00000100:00100000:3.0:1789760765.772898:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565353856:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760765.845826:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760765.845850:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565354368:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760765.859888:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983913 gen 3 00000100:00100000:3.0:1789760765.879413:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565354624:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760765.934124:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565355136:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760765.945163:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983919 gen 3 00000008:00010000:3.0:1789760765.945285:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760765.945292:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760765.945302:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760765.945373:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760765.945411:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565355136:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760765.946905:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565355264:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760765.957532:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983920 gen 3 00000008:00010000:3.0:1789760765.957620:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760765.957631:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760765.957686:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760765.957710:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760765.957741:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565355264:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760765.959422:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760765.959447:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565355392:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760765.969986:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983921 gen 3 00000008:00010000:3.0:1789760765.970036:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760765.970046:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760765.970058:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760765.970091:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760765.970121:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565355392:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760765.986218:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760765.986245:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565355648:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760765.999185:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983923 gen 3 00000008:00010000:3.0:1789760765.999236:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760765.999245:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760765.999256:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760765.999390:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760765.999423:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565355648:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.001465:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760766.001491:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565355776:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.012429:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983924 gen 3 00000008:00010000:3.0:1789760766.012483:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189540 00000020:00200000:3.0:1789760766.012491:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.012502:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.012619:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189540 00000100:00100000:3.0:1789760766.012678:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565355776:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.015979:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760766.016446:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565355904:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.031439:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983925 gen 3 00000008:00010000:3.0:1789760766.031498:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189540 00000020:00200000:3.0:1789760766.031507:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.031519:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.031552:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189540 00000100:00100000:3.0:1789760766.031801:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565355904:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.033072:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565356032:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.042888:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983926 gen 3 00000008:00010000:3.0:1789760766.042938:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760766.043014:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.043026:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.043110:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760766.043140:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565356032:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.077227:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565356416:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.086150:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983929 gen 3 00000008:00010000:3.0:1789760766.086198:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760766.086206:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.086217:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.086255:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760766.086285:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565356416:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760766.098198:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16622, 16622] 00000080:00200000:3.0:1789760766.098265:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760766.098271:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760766.098286:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68087808 i_blocks 133008 00000020:00200000:3.0:1789760766.098290:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760766.098298:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760766.098346:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68087808, count: 4096 00000080:00200000:3.0:1789760766.098355:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68087808, bytes: 4096 00000080:00200000:3.0:1789760766.098365:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760766.098373:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760766.098376:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68087808, 68091904) 0 0 00000080:00200000:3.0:1789760766.098381:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760766.098397:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16623, 16623] 00020000:00200000:3.0:1789760766.098406:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68087808 chunk: [68087808, 68091904) 68091904, 4096 00020000:00200000:3.0:1789760766.098411:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760766.098430:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16623, 16623] 00000008:00010000:3.0:1789760766.098447:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760766.098453:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760766.098463:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68087808, 68091904) append 0 append lockpos 0 00000080:00200000:3.0:1789760766.098477:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68087808=0x40ef000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760766.098555:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760766.098558:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68091904 00000020:00010000:3.0:1789760766.098567:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0f38 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760766.098570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0f38 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.098573:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.098576:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0f38 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760766.098579:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760766.098588:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760766.099244:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760766.099266:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760766.099392:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 68087808<>68091904 00000080:00200000:3.0:1789760766.111156:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16623, 16623] 00000080:00200000:3.0:1789760766.111174:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760766.111180:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760766.111196:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68091904 i_blocks 133016 00000020:00200000:3.0:1789760766.111200:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760766.111224:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760766.111258:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68091904, count: 4096 00000080:00200000:3.0:1789760766.111265:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68091904, bytes: 4096 00000080:00200000:3.0:1789760766.111280:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760766.111289:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760766.111292:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68091904, 68096000) 0 0 00000080:00200000:3.0:1789760766.111296:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760766.111314:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16624, 16624] 00020000:00200000:3.0:1789760766.111324:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68091904 chunk: [68091904, 68096000) 68096000, 4096 00020000:00200000:3.0:1789760766.111329:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760766.111349:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16624, 16624] 00000008:00010000:3.0:1789760766.111370:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760766.111378:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760766.111392:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68091904, 68096000) append 0 append lockpos 0 00000080:00200000:3.0:1789760766.111408:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68091904=0x40f0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760766.111473:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760766.111477:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68096000 00000020:00010000:3.0:1789760766.111486:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0f38 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760766.111489:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0f38 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.111493:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.111497:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0f38 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760766.111500:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760766.111509:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760766.111516:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760766.111534:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760766.112008:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 68091904<>68096000 00000080:00200000:3.0:1789760766.125723:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16624, 16624] 00000080:00200000:3.0:1789760766.125744:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760766.125751:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760766.125767:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68096000 i_blocks 133024 00000020:00200000:3.0:1789760766.125772:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760766.125783:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760766.126004:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68096000, count: 4096 00000080:00200000:3.0:1789760766.126014:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68096000, bytes: 4096 00000080:00200000:3.0:1789760766.126029:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760766.126038:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760766.126040:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68096000, 68100096) 0 0 00000080:00200000:3.0:1789760766.126045:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760766.126061:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16625, 16625] 00020000:00200000:3.0:1789760766.126070:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68096000 chunk: [68096000, 68100096) 68100096, 4096 00020000:00200000:3.0:1789760766.126075:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760766.126095:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16625, 16625] 00000008:00010000:3.0:1789760766.126112:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760766.126118:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760766.126131:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68096000, 68100096) append 0 append lockpos 0 00000080:00200000:3.0:1789760766.126145:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68096000=0x40f1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760766.126794:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760766.126801:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68100096 00000020:00010000:3.0:1789760766.126811:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0f38 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760766.126816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0f38 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.126819:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.126822:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0f38 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760766.126825:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760766.126836:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760766.126843:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760766.126867:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760766.127006:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 68096000<>68100096 00000080:00200000:3.0:1789760766.137678:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16625, 16625] 00000080:00200000:3.0:1789760766.137695:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760766.137704:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760766.150827:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16626, 16626] 00000080:00200000:3.0:1789760766.150844:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760766.150851:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760766.150866:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68104192 i_blocks 133040 00000020:00200000:3.0:1789760766.150871:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760766.150880:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000100:00100000:3.0:1789760766.151871:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760766.151894:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565357184:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.167049:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983935 gen 3 00000100:00100000:3.0:1789760766.191205:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760766.191416:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565357440:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.202744:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983937 gen 3 00000008:00010000:3.0:1789760766.202798:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760766.202807:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.202819:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.202848:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760766.202880:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565357440:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760766.213871:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16630, 16630] 00000080:00200000:3.0:1789760766.213923:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760766.213929:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760766.213954:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68120576 i_blocks 133072 00000020:00200000:3.0:1789760766.213958:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760766.213968:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760766.214014:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68120576, count: 4096 00000080:00200000:3.0:1789760766.214023:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68120576, bytes: 4096 00000080:00200000:3.0:1789760766.214034:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760766.214042:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760766.214044:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68120576, 68124672) 0 0 00000080:00200000:3.0:1789760766.214048:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760766.214065:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16631, 16631] 00020000:00200000:3.0:1789760766.214074:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68120576 chunk: [68120576, 68124672) 68124672, 4096 00020000:00200000:3.0:1789760766.214078:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760766.214099:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16631, 16631] 00000008:00010000:3.0:1789760766.214117:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760766.214123:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760766.214135:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68120576, 68124672) append 0 append lockpos 0 00000080:00200000:3.0:1789760766.214150:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68120576=0x40f7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760766.214847:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760766.214853:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68124672 00000020:00010000:3.0:1789760766.214863:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760766.214867:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.214870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.214874:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760766.214877:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760766.214886:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760766.214894:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760766.215067:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760766.215201:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 68120576<>68124672 00000100:00100000:3.0:1789760766.215248:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760766.215269:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565357696:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.229597:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983939 gen 3 00000008:00010000:3.0:1789760766.229813:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:3.0:1789760766.229822:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.229841:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.229855:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:3.0:1789760766.229884:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565357696:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.250187:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565357952:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.261314:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983941 gen 3 00000008:00010000:3.0:1789760766.261368:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760766.261377:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.261388:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.261416:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760766.261447:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565357952:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760766.262463:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189a80 00000008:00100000:3.0:1789760766.262794:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 68132864<>68136960 00000100:00100000:3.0:1789760766.262917:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760766.262938:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565358080:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760766.493532:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760766.572405:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760766.572496:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565360640:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.581013:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983959 gen 3 00000008:00010000:3.0:1789760766.581065:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760766.581074:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.581084:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.581108:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760766.581136:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565360640:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.583844:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565360768:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.598403:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983960 gen 3 00000008:00010000:3.0:1789760766.598462:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760766.598472:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.598493:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.598516:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760766.599116:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565360768:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760766.599162:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189540 00000008:00100000:3.0:1789760766.599285:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 68210688<>68214784 00000100:00100000:3.0:1789760766.599332:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760766.599346:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565360896:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.615930:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983961 gen 3 00000008:00010000:3.0:1789760766.615988:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189540 00000020:00200000:3.0:1789760766.615997:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.616008:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.616084:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189540 00000100:00100000:3.0:1789760766.616112:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565360896:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.619038:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565361024:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.631994:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983962 gen 3 00000008:00010000:3.0:1789760766.632048:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189540 00000020:00200000:3.0:1789760766.632057:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.632068:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.632092:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189540 00000100:00100000:3.0:1789760766.632121:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565361024:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.633254:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565361152:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.644972:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983963 gen 3 00000100:00100000:3.0:1789760766.704078:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565361792:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.783350:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565362304:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.799344:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983972 gen 3 00000008:00010000:3.0:1789760766.799393:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760766.799402:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.799413:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.799441:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760766.799475:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565362304:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760766.869571:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16667, 16667] 00000080:00200000:3.0:1789760766.869589:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760766.869605:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760766.869621:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68272128 i_blocks 133368 00000020:00200000:3.0:1789760766.869626:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760766.869639:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760766.869696:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68272128, count: 4096 00000080:00200000:3.0:1789760766.869703:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68272128, bytes: 4096 00000080:00200000:3.0:1789760766.869713:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760766.869722:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760766.869725:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68272128, 68276224) 0 0 00000080:00200000:3.0:1789760766.869729:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760766.869747:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16668, 16668] 00020000:00200000:3.0:1789760766.869756:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68272128 chunk: [68272128, 68276224) 68276224, 4096 00020000:00200000:3.0:1789760766.869761:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760766.869861:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16668, 16668] 00000008:00010000:3.0:1789760766.869878:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760766.869884:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760766.869897:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68272128, 68276224) append 0 append lockpos 0 00000080:00200000:3.0:1789760766.869911:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68272128=0x411c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760766.869968:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760766.869971:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68276224 00000020:00010000:3.0:1789760766.869979:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760766.869982:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.869985:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.869988:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760766.869991:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760766.869999:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760766.870005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760766.870022:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760766.870142:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 68272128<>68276224 00000080:00200000:3.0:1789760766.881978:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16668, 16668] 00000080:00200000:3.0:1789760766.881997:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760766.882003:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760766.882019:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68276224 i_blocks 133376 00000020:00200000:3.0:1789760766.882023:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760766.882035:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760766.882070:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68276224, count: 4096 00000080:00200000:3.0:1789760766.882077:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68276224, bytes: 4096 00000080:00200000:3.0:1789760766.882088:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760766.882098:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760766.882101:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68276224, 68280320) 0 0 00000080:00200000:3.0:1789760766.882106:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760766.882123:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16669, 16669] 00020000:00200000:3.0:1789760766.882134:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68276224 chunk: [68276224, 68280320) 68280320, 4096 00020000:00200000:3.0:1789760766.882140:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760766.882159:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16669, 16669] 00000008:00010000:3.0:1789760766.882178:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760766.882185:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760766.882197:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68276224, 68280320) append 0 append lockpos 0 00000080:00200000:3.0:1789760766.882211:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68276224=0x411d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760766.882268:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760766.882272:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68280320 00000020:00010000:3.0:1789760766.882280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760766.882284:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.882287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.882291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760766.882294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760766.882302:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760766.882309:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760766.882326:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760766.882460:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 68276224<>68280320 00000100:00100000:3.0:1789760766.882510:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565362944:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.893400:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983977 gen 3 00000008:00010000:3.0:1789760766.893453:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760766.893462:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.893474:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.893617:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760766.893828:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565362944:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.895154:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565363072:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.906145:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983978 gen 3 00000008:00010000:3.0:1789760766.906255:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760766.906263:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.906274:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.906339:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760766.906370:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565363072:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.919680:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565363328:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.929895:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983980 gen 3 00000008:00010000:3.0:1789760766.929959:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760766.929969:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.929981:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.930009:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760766.930041:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565363328:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.931014:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565363456:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760766.947397:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16673, 16673] 00000080:00200000:3.0:1789760766.947416:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760766.947422:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760766.947440:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68296704 i_blocks 133416 00000020:00200000:3.0:1789760766.947445:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760766.947455:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760766.947488:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68296704, count: 4096 00000080:00200000:3.0:1789760766.947500:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68296704, bytes: 4096 00000080:00200000:3.0:1789760766.947515:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760766.947524:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760766.947526:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68296704, 68300800) 0 0 00000080:00200000:3.0:1789760766.947531:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760766.947548:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16674, 16674] 00020000:00200000:3.0:1789760766.947613:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68296704 chunk: [68296704, 68300800) 68300800, 4096 00020000:00200000:3.0:1789760766.947618:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760766.947640:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16674, 16674] 00000008:00010000:3.0:1789760766.947794:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760766.947801:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760766.947822:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68296704, 68300800) append 0 append lockpos 0 00000080:00200000:3.0:1789760766.947837:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68296704=0x4122000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760766.947902:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760766.947905:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68300800 00000020:00010000:3.0:1789760766.947913:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760766.947917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.947920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.947923:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760766.947926:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760766.947934:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760766.948041:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760766.948066:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760766.948381:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 68296704<>68300800 00000100:00100000:3.0:1789760766.949775:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760766.949809:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565363584:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.960953:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983982 gen 3 00000008:00010000:3.0:1789760766.961004:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760766.961100:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.961114:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.961195:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760766.961227:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565363584:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.962137:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565363712:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760766.975272:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983983 gen 3 00000008:00010000:3.0:1789760766.975323:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760766.975332:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760766.975344:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760766.975459:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760766.975614:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565363712:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760766.990473:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16676, 16676] 00000080:00200000:3.0:1789760766.990499:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760766.990506:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760766.990523:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68308992 i_blocks 133440 00000020:00200000:3.0:1789760766.990527:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760766.991102:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760766.991141:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68308992, count: 4096 00000080:00200000:3.0:1789760766.991149:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68308992, bytes: 4096 00000080:00200000:3.0:1789760766.991160:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760766.991169:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760766.991172:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68308992, 68313088) 0 0 00000080:00200000:3.0:1789760766.991178:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760766.991196:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16677, 16677] 00020000:00200000:3.0:1789760766.991206:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68308992 chunk: [68308992, 68313088) 68313088, 4096 00020000:00200000:3.0:1789760766.991210:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760766.991230:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16677, 16677] 00000008:00010000:3.0:1789760766.991248:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760766.991255:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760766.991267:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68308992, 68313088) append 0 append lockpos 0 00000080:00200000:3.0:1789760766.991282:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68308992=0x4125000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760766.991347:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760766.991351:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68313088 00000020:00010000:3.0:1789760766.991359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760766.991363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.991366:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760766.991370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760766.991373:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760766.991381:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760766.991389:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760766.991410:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760766.992030:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 68308992<>68313088 00000100:00100000:3.0:1789760767.232547:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760767.232564:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565365760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.251131:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294983999 gen 3 00000008:00010000:3.0:1789760767.251228:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760767.251234:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760767.251243:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760767.251261:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760767.251280:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565365760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.252684:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760767.252703:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565365888:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.274252:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984000 gen 3 00000008:00010000:3.0:1789760767.274289:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760767.274295:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760767.274303:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760767.274325:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760767.274347:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565365888:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.275486:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565366016:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.294046:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984001 gen 3 00000008:00010000:3.0:1789760767.294084:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760767.294090:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760767.294099:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760767.294121:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760767.294144:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565366016:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.295159:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565366144:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.381314:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760767.381345:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565366656:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.393467:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984006 gen 3 00000008:00010000:3.0:1789760767.393628:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760767.393638:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760767.393693:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760767.393716:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760767.393747:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565366656:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.394854:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760767.394937:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565366784:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.403213:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984007 gen 3 00000008:00010000:3.0:1789760767.403264:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760767.403273:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760767.403284:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760767.403312:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760767.403343:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565366784:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.404558:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760767.404578:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565366912:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.413047:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984008 gen 3 00000008:00010000:3.0:1789760767.413100:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760767.413108:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760767.413119:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760767.413144:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760767.413173:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565366912:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.414400:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760767.414421:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565367040:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.428377:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984009 gen 3 00000008:00010000:3.0:1789760767.428428:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760767.428437:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760767.428449:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760767.428474:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760767.428504:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565367040:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760767.430261:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189e40 00000008:00100000:3.0:1789760767.431243:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 68411392<>68415488 00000080:00200000:3.0:1789760767.508101:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16706, 16706] 00000080:00200000:3.0:1789760767.508120:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.508128:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.508144:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68431872 i_blocks 133680 00000020:00200000:3.0:1789760767.508149:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.508160:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.508194:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68431872, count: 4096 00000080:00200000:3.0:1789760767.508202:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68431872, bytes: 4096 00000080:00200000:3.0:1789760767.508214:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.508223:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.508226:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68431872, 68435968) 0 0 00000080:00200000:3.0:1789760767.508230:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.508250:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16707, 16707] 00020000:00200000:3.0:1789760767.508260:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68431872 chunk: [68431872, 68435968) 68435968, 4096 00020000:00200000:3.0:1789760767.508264:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.508285:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16707, 16707] 00000008:00010000:3.0:1789760767.508305:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.508311:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.508324:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68431872, 68435968) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.508338:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68431872=0x4143000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.508400:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.508403:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68435968 00000020:00010000:3.0:1789760767.508412:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.508416:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.508419:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.508423:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.508426:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.508435:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.508443:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.508466:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760767.508798:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 68431872<>68435968 00010000:00010000:3.0:1789760767.516039:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1789760767.518682:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16707, 16707] 00000080:00200000:3.0:1789760767.518700:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.518706:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.518723:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68435968 i_blocks 133688 00000020:00200000:3.0:1789760767.518727:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.518737:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.518770:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68435968, count: 4096 00000080:00200000:3.0:1789760767.518778:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68435968, bytes: 4096 00000080:00200000:3.0:1789760767.518790:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.518799:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.518801:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68435968, 68440064) 0 0 00000080:00200000:3.0:1789760767.518807:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.518825:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16708, 16708] 00020000:00200000:3.0:1789760767.518833:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68435968 chunk: [68435968, 68440064) 68440064, 4096 00020000:00200000:3.0:1789760767.518838:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.518859:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16708, 16708] 00000008:00010000:3.0:1789760767.518888:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.518894:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.518906:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68435968, 68440064) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.518921:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68435968=0x4144000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.519103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.519106:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68440064 00000020:00010000:3.0:1789760767.519115:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.519119:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.519122:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.519126:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.519129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.519137:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.519144:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.519162:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760767.519285:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 68435968<>68440064 00000080:00200000:3.0:1789760767.533296:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16708, 16708] 00000080:00200000:3.0:1789760767.533319:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.533326:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.533344:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68440064 i_blocks 133696 00000020:00200000:3.0:1789760767.533348:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.533362:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.533398:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68440064, count: 4096 00000080:00200000:3.0:1789760767.533408:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68440064, bytes: 4096 00000080:00200000:3.0:1789760767.533421:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.533431:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.533433:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68440064, 68444160) 0 0 00000080:00200000:3.0:1789760767.533438:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.533457:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16709, 16709] 00020000:00200000:3.0:1789760767.533466:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68440064 chunk: [68440064, 68444160) 68444160, 4096 00020000:00200000:3.0:1789760767.533470:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.533505:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16709, 16709] 00000008:00010000:3.0:1789760767.533525:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.533539:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.533553:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68440064, 68444160) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.533567:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68440064=0x4145000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.533671:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.533675:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68444160 00000020:00010000:3.0:1789760767.533683:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.533687:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.533690:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.533693:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.533696:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.533705:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.533712:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.533733:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760767.533874:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 68440064<>68444160 00000080:00200000:3.0:1789760767.550439:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16709, 16709] 00000080:00200000:3.0:1789760767.550456:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.550461:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.550477:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68444160 i_blocks 133704 00000020:00200000:3.0:1789760767.550481:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.550499:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.550534:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68444160, count: 4096 00000080:00200000:3.0:1789760767.550541:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68444160, bytes: 4096 00000080:00200000:3.0:1789760767.550553:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.550562:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.550565:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68444160, 68448256) 0 0 00000080:00200000:3.0:1789760767.550571:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.550588:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16710, 16710] 00020000:00200000:3.0:1789760767.550605:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68444160 chunk: [68444160, 68448256) 68448256, 4096 00020000:00200000:3.0:1789760767.550610:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.550630:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16710, 16710] 00000008:00010000:3.0:1789760767.550698:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.550704:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.550717:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68444160, 68448256) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.550731:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68444160=0x4146000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.550792:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.550795:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68448256 00000020:00010000:3.0:1789760767.550804:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.550808:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.550811:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.550815:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.550818:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.550826:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.550833:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.550850:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760767.550989:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 68444160<>68448256 00000080:00200000:3.0:1789760767.562549:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16710, 16710] 00000080:00200000:3.0:1789760767.562568:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.562574:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.562598:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68448256 i_blocks 133712 00000020:00200000:3.0:1789760767.562603:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.562613:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.562693:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68448256, count: 4096 00000080:00200000:3.0:1789760767.562701:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68448256, bytes: 4096 00000080:00200000:3.0:1789760767.562713:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.562720:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.562723:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68448256, 68452352) 0 0 00000080:00200000:3.0:1789760767.562728:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.562744:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16711, 16711] 00020000:00200000:3.0:1789760767.562753:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68448256 chunk: [68448256, 68452352) 68452352, 4096 00020000:00200000:3.0:1789760767.562757:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.562775:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16711, 16711] 00000008:00010000:3.0:1789760767.562794:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.562801:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.562813:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68448256, 68452352) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.562827:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68448256=0x4147000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.562890:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.562893:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68452352 00000020:00010000:3.0:1789760767.562902:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.562906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.562908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.562912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.562915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.562924:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.562931:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.562951:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760767.563082:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 68448256<>68452352 00000080:00200000:3.0:1789760767.577208:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16711, 16711] 00000080:00200000:3.0:1789760767.577225:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.577230:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760767.578924:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760767.580952:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565368448:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.590142:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984020 gen 3 00000008:00010000:3.0:1789760767.590190:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760767.590197:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760767.590206:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760767.590278:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760767.590303:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565368448:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760767.625292:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16714, 16714] 00000080:00200000:3.0:1789760767.625311:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.625327:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.625342:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68464640 i_blocks 133744 00000020:00200000:3.0:1789760767.625346:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.625356:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.625424:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68464640, count: 4096 00000080:00200000:3.0:1789760767.625433:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68464640, bytes: 4096 00000080:00200000:3.0:1789760767.625445:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.625453:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.625456:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68464640, 68468736) 0 0 00000080:00200000:3.0:1789760767.625460:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.625477:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16715, 16715] 00020000:00200000:3.0:1789760767.625831:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68464640 chunk: [68464640, 68468736) 68468736, 4096 00020000:00200000:3.0:1789760767.625837:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.625859:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16715, 16715] 00000008:00010000:3.0:1789760767.625960:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.625966:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.625977:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68464640, 68468736) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.625991:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68464640=0x414b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.626055:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.626058:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68468736 00000020:00010000:3.0:1789760767.626067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.626071:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.626074:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.626077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.626080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.626088:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.626095:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.626115:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760767.626240:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 68464640<>68468736 00000100:00100000:3.0:1789760767.626294:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760767.626314:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565368832:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.667158:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565369216:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.791725:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565370112:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760767.850488:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16728, 16728] 00000080:00200000:3.0:1789760767.850506:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.850511:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.850526:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68521984 i_blocks 133856 00000020:00200000:3.0:1789760767.850530:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.850540:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.850576:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68521984, count: 4096 00000080:00200000:3.0:1789760767.850584:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68521984, bytes: 4096 00000080:00200000:3.0:1789760767.850603:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.850613:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.850616:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68521984, 68526080) 0 0 00000080:00200000:3.0:1789760767.850620:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.850639:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16729, 16729] 00020000:00200000:3.0:1789760767.850694:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68521984 chunk: [68521984, 68526080) 68526080, 4096 00020000:00200000:3.0:1789760767.850699:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.850720:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16729, 16729] 00000008:00010000:3.0:1789760767.850739:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.850745:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.850756:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68521984, 68526080) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.850770:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68521984=0x4159000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.855289:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.855297:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68526080 00000020:00010000:3.0:1789760767.855314:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.855319:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.855323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.855328:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.855331:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.855343:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.855352:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.855486:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760767.855636:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 68521984<>68526080 00000100:00100000:3.0:1789760767.855690:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760767.855709:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565370624:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.870104:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984037 gen 3 00000008:00010000:3.0:1789760767.870150:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:3.0:1789760767.870159:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760767.870170:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760767.870194:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:3.0:1789760767.870223:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565370624:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760767.882682:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565370880:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760767.895086:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16731, 16731] 00000080:00200000:3.0:1789760767.895102:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.895109:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.895123:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68534272 i_blocks 133880 00000020:00200000:3.0:1789760767.895127:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.895136:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.895169:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68534272, count: 4096 00000080:00200000:3.0:1789760767.895176:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68534272, bytes: 4096 00000080:00200000:3.0:1789760767.895185:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.895194:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.895196:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68534272, 68538368) 0 0 00000080:00200000:3.0:1789760767.895200:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.895217:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16732, 16732] 00020000:00200000:3.0:1789760767.895240:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68534272 chunk: [68534272, 68538368) 68538368, 4096 00020000:00200000:3.0:1789760767.895245:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.895266:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16732, 16732] 00000008:00010000:3.0:1789760767.895283:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.895289:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.895301:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68534272, 68538368) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.895314:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68534272=0x415c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.895374:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.895377:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68538368 00000020:00010000:3.0:1789760767.895385:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.895389:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.895392:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.895396:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.895399:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.895407:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.895414:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.895430:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760767.895554:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 68534272<>68538368 00000080:00200000:3.0:1789760767.908245:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16732, 16732] 00000080:00200000:3.0:1789760767.908262:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.908268:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.908283:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68538368 i_blocks 133888 00000020:00200000:3.0:1789760767.908286:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.908295:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.908492:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68538368, count: 4096 00000080:00200000:3.0:1789760767.908501:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68538368, bytes: 4096 00000080:00200000:3.0:1789760767.908514:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.908522:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.908582:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68538368, 68542464) 0 0 00000080:00200000:3.0:1789760767.908596:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.908614:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16733, 16733] 00020000:00200000:3.0:1789760767.908623:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68538368 chunk: [68538368, 68542464) 68542464, 4096 00020000:00200000:3.0:1789760767.908627:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.908668:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16733, 16733] 00000008:00010000:3.0:1789760767.908687:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.908767:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.908782:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68538368, 68542464) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.908797:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68538368=0x415d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.908858:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.908861:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68542464 00000020:00010000:3.0:1789760767.908871:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.908875:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.908878:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.908882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188038 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.908885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.908894:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.908901:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.908920:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760767.909211:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 68538368<>68542464 00000080:00200000:3.0:1789760767.924276:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16733, 16733] 00000080:00200000:3.0:1789760767.924294:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.924300:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.936801:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16734, 16734] 00000080:00200000:3.0:1789760767.937011:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.937018:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.937099:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68546560 i_blocks 133904 00000020:00200000:3.0:1789760767.937104:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.937116:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.937179:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68546560, count: 4096 00000080:00200000:3.0:1789760767.937189:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68546560, bytes: 4096 00000080:00200000:3.0:1789760767.937201:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.937209:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.937212:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68546560, 68550656) 0 0 00000080:00200000:3.0:1789760767.937217:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.937234:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16735, 16735] 00020000:00200000:3.0:1789760767.937243:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68546560 chunk: [68546560, 68550656) 68550656, 4096 00020000:00200000:3.0:1789760767.937248:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.937268:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16735, 16735] 00000008:00010000:3.0:1789760767.937287:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.937293:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.937305:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68546560, 68550656) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.937320:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68546560=0x415f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.937381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.937384:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68550656 00000020:00010000:3.0:1789760767.937394:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.937397:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.937400:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.937403:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.937406:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.938010:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.938021:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.938051:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760767.938185:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 68546560<>68550656 00000080:00200000:3.0:1789760767.955054:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16735, 16735] 00000080:00200000:3.0:1789760767.955070:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.955076:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760767.955973:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565371520:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760767.999391:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16738, 16738] 00000080:00200000:3.0:1789760767.999407:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760767.999423:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760767.999439:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68562944 i_blocks 133936 00000020:00200000:3.0:1789760767.999443:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760767.999452:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760767.999511:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68562944, count: 4096 00000080:00200000:3.0:1789760767.999519:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68562944, bytes: 4096 00000080:00200000:3.0:1789760767.999529:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760767.999537:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760767.999539:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68562944, 68567040) 0 0 00000080:00200000:3.0:1789760767.999543:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760767.999560:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16739, 16739] 00020000:00200000:3.0:1789760767.999568:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68562944 chunk: [68562944, 68567040) 68567040, 4096 00020000:00200000:3.0:1789760767.999573:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760767.999601:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16739, 16739] 00000008:00010000:3.0:1789760767.999618:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760767.999624:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760767.999638:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68562944, 68567040) append 0 append lockpos 0 00000080:00200000:3.0:1789760767.999706:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68562944=0x4163000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760767.999770:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760767.999773:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68567040 00000020:00010000:3.0:1789760767.999782:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760767.999786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.999790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760767.999794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760767.999798:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760767.999807:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760767.999815:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760767.999842:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760767.999967:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 68562944<>68567040 00000100:00100000:3.0:1789760768.000008:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760768.000027:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565371904:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.008677:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984047 gen 3 00000008:00010000:3.0:1789760768.008727:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:3.0:1789760768.008736:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.008747:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.008771:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:3.0:1789760768.008801:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565371904:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760768.026190:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16740, 16740] 00000080:00200000:3.0:1789760768.026217:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.027190:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.027213:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68571136 i_blocks 133952 00000020:00200000:3.0:1789760768.027219:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.027245:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.027595:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68571136, count: 4096 00000080:00200000:3.0:1789760768.027607:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68571136, bytes: 4096 00000080:00200000:3.0:1789760768.027623:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.027633:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.027635:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68571136, 68575232) 0 0 00000080:00200000:3.0:1789760768.027641:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.028754:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16741, 16741] 00020000:00200000:3.0:1789760768.028765:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68571136 chunk: [68571136, 68575232) 68575232, 4096 00020000:00200000:3.0:1789760768.028769:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.028795:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16741, 16741] 00000008:00010000:3.0:1789760768.028815:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.028820:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.028832:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68571136, 68575232) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.028847:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68571136=0x4165000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.028919:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.028922:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68575232 00000020:00010000:3.0:1789760768.028931:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.028935:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.028939:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.028942:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.028945:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.028954:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.028961:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.028991:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760768.029135:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 68571136<>68575232 00000100:00100000:3.0:1789760768.029193:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760768.029217:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565372160:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.042829:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984049 gen 3 00000008:00010000:3.0:1789760768.042884:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:3.0:1789760768.042892:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.042904:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.042979:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:3.0:1789760768.043009:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565372160:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.047884:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565372288:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.068193:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984050 gen 3 00000008:00010000:3.0:1789760768.068244:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189540 00000020:00200000:3.0:1789760768.068253:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.068263:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.068290:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189540 00000100:00100000:3.0:1789760768.068319:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565372288:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760768.080678:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16743, 16743] 00000080:00200000:3.0:1789760768.080694:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.080700:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.080725:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68583424 i_blocks 133976 00000020:00200000:3.0:1789760768.080729:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.080742:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.080776:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68583424, count: 4096 00000080:00200000:3.0:1789760768.080784:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68583424, bytes: 4096 00000080:00200000:3.0:1789760768.080795:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.080803:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.080806:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68583424, 68587520) 0 0 00000080:00200000:3.0:1789760768.080810:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.080826:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16744, 16744] 00020000:00200000:3.0:1789760768.080835:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68583424 chunk: [68583424, 68587520) 68587520, 4096 00020000:00200000:3.0:1789760768.080840:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.080857:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16744, 16744] 00000008:00010000:3.0:1789760768.080874:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.080881:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.080892:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68583424, 68587520) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.080906:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68583424=0x4168000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.080960:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.080963:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68587520 00000020:00010000:3.0:1789760768.080971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.080975:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.080977:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.080981:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.080984:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.080992:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.080998:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.081015:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760768.081149:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 68583424<>68587520 00000100:00100000:3.0:1789760768.081195:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760768.081213:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565372544:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.119195:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565372800:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.183516:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565373184:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.197354:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984057 gen 3 00000008:00010000:3.0:1789760768.197419:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760768.197428:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.197440:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.197463:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760768.197502:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565373184:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.217595:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565373440:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.232717:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984059 gen 3 00000008:00010000:3.0:1789760768.232768:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760768.232775:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.232786:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.232911:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760768.232943:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565373440:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.234251:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565373568:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.273239:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565373824:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.289450:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984062 gen 3 00000008:00010000:3.0:1789760768.289504:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760768.289512:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.289523:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.289615:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760768.289684:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565373824:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760768.312311:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16755, 16755] 00000080:00200000:3.0:1789760768.312329:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.312336:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.312353:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68632576 i_blocks 134072 00000020:00200000:3.0:1789760768.312358:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.312371:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.312405:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68632576, count: 4096 00000080:00200000:3.0:1789760768.312412:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68632576, bytes: 4096 00000080:00200000:3.0:1789760768.312424:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.312433:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.312435:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68632576, 68636672) 0 0 00000080:00200000:3.0:1789760768.312440:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.312457:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16756, 16756] 00020000:00200000:3.0:1789760768.312466:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68632576 chunk: [68632576, 68636672) 68636672, 4096 00020000:00200000:3.0:1789760768.312471:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.312490:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16756, 16756] 00000008:00010000:3.0:1789760768.312508:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.312514:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.312527:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68632576, 68636672) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.312541:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68632576=0x4174000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.312617:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.312620:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68636672 00000020:00010000:3.0:1789760768.312630:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.312634:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.312637:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.312640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.312741:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.312751:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.312759:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.312778:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760768.312910:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 68632576<>68636672 00000080:00200000:3.0:1789760768.332000:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16756, 16756] 00000080:00200000:3.0:1789760768.332018:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.332024:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.332041:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68636672 i_blocks 134080 00000020:00200000:3.0:1789760768.332045:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.332055:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.332091:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68636672, count: 4096 00000080:00200000:3.0:1789760768.332098:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68636672, bytes: 4096 00000080:00200000:3.0:1789760768.332108:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.332116:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.332119:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68636672, 68640768) 0 0 00000080:00200000:3.0:1789760768.332123:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.332212:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16757, 16757] 00020000:00200000:3.0:1789760768.332221:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68636672 chunk: [68636672, 68640768) 68640768, 4096 00020000:00200000:3.0:1789760768.332226:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.332248:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16757, 16757] 00000008:00010000:3.0:1789760768.332267:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.332273:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.332285:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68636672, 68640768) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.332299:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68636672=0x4175000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.332362:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.332364:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68640768 00000020:00010000:3.0:1789760768.332372:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.332376:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.332379:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.332382:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.332386:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.332394:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.332401:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.332419:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760768.332538:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 68636672<>68640768 00000080:00200000:3.0:1789760768.357438:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16757, 16757] 00000080:00200000:3.0:1789760768.357455:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.357461:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.357476:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68640768 i_blocks 134088 00000020:00200000:3.0:1789760768.357481:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.357491:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.357541:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68640768, count: 4096 00000080:00200000:3.0:1789760768.357549:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68640768, bytes: 4096 00000080:00200000:3.0:1789760768.357565:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.357574:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.357577:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68640768, 68644864) 0 0 00000080:00200000:3.0:1789760768.357582:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.357611:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16758, 16758] 00020000:00200000:3.0:1789760768.357620:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68640768 chunk: [68640768, 68644864) 68644864, 4096 00020000:00200000:3.0:1789760768.357625:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.357676:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16758, 16758] 00000008:00010000:3.0:1789760768.357696:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.357703:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.357715:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68640768, 68644864) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.358011:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68640768=0x4176000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.358076:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.358079:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68644864 00000020:00010000:3.0:1789760768.358088:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.358092:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.358096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.358100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.358103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.358113:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.358121:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.358142:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760768.358265:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 68640768<>68644864 00000080:00200000:3.0:1789760768.377836:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16758, 16758] 00000080:00200000:3.0:1789760768.377868:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.377875:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.377897:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68644864 i_blocks 134096 00000020:00200000:3.0:1789760768.377903:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.377921:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.377959:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68644864, count: 4096 00000080:00200000:3.0:1789760768.377968:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68644864, bytes: 4096 00000080:00200000:3.0:1789760768.377984:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.377993:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.377995:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68644864, 68648960) 0 0 00000080:00200000:3.0:1789760768.377999:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.378016:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16759, 16759] 00020000:00200000:3.0:1789760768.378025:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68644864 chunk: [68644864, 68648960) 68648960, 4096 00020000:00200000:3.0:1789760768.378030:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.378053:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16759, 16759] 00000008:00010000:3.0:1789760768.378071:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.378077:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.378089:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68644864, 68648960) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.378105:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68644864=0x4177000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.378179:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.378181:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68648960 00000020:00010000:3.0:1789760768.378189:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.378193:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.378196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.378200:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.378203:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.378212:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.378219:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.378251:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760768.378384:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 68644864<>68648960 00000080:00200000:3.0:1789760768.398126:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16759, 16759] 00000080:00200000:3.0:1789760768.398142:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.398149:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.398166:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68648960 i_blocks 134104 00000020:00200000:3.0:1789760768.398170:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.398180:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.398213:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68648960, count: 4096 00000080:00200000:3.0:1789760768.398221:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68648960, bytes: 4096 00000080:00200000:3.0:1789760768.398351:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.398362:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.398365:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68648960, 68653056) 0 0 00000080:00200000:3.0:1789760768.398369:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.398386:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16760, 16760] 00020000:00200000:3.0:1789760768.398395:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68648960 chunk: [68648960, 68653056) 68653056, 4096 00020000:00200000:3.0:1789760768.398400:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.398420:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16760, 16760] 00000008:00010000:3.0:1789760768.398439:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.398445:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.398456:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68648960, 68653056) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.398470:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68648960=0x4178000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.398615:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.398618:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68653056 00000020:00010000:3.0:1789760768.398628:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.398632:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.398635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.398639:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.398669:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.398681:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.398757:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.398777:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760768.398906:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 68648960<>68653056 00000080:00200000:3.0:1789760768.415614:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16760, 16760] 00000080:00200000:3.0:1789760768.415725:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.415733:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.415752:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68653056 i_blocks 134112 00000020:00200000:3.0:1789760768.415758:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.415774:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.415808:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68653056, count: 4096 00000080:00200000:3.0:1789760768.415892:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68653056, bytes: 4096 00000080:00200000:3.0:1789760768.415909:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.415919:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.415922:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68653056, 68657152) 0 0 00000080:00200000:3.0:1789760768.415927:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.415946:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16761, 16761] 00020000:00200000:3.0:1789760768.415955:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68653056 chunk: [68653056, 68657152) 68657152, 4096 00020000:00200000:3.0:1789760768.415960:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.415984:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16761, 16761] 00000008:00010000:3.0:1789760768.416002:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.416008:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.416020:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68653056, 68657152) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.416034:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68653056=0x4179000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.416111:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.416114:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68657152 00000020:00010000:3.0:1789760768.416124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.416127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.416130:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.416134:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.416137:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.416147:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.416154:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.416184:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760768.416540:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 68653056<>68657152 00000080:00200000:3.0:1789760768.439472:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16761, 16761] 00000080:00200000:3.0:1789760768.439488:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.439494:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.439510:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68657152 i_blocks 134120 00000020:00200000:3.0:1789760768.439514:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.439524:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.439559:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68657152, count: 4096 00000080:00200000:3.0:1789760768.439567:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68657152, bytes: 4096 00000080:00200000:3.0:1789760768.439577:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.439596:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.439598:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68657152, 68661248) 0 0 00000080:00200000:3.0:1789760768.439603:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.439621:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16762, 16762] 00020000:00200000:3.0:1789760768.439629:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68657152 chunk: [68657152, 68661248) 68661248, 4096 00020000:00200000:3.0:1789760768.439634:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.439703:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16762, 16762] 00000008:00010000:3.0:1789760768.439722:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.439728:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.439740:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68657152, 68661248) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.439754:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68657152=0x417a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.439808:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.439931:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68661248 00000020:00010000:3.0:1789760768.439941:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.439944:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.439947:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.439952:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.439954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.439964:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.439971:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.439990:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760768.440117:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 68657152<>68661248 00000080:00200000:3.0:1789760768.449347:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16762, 16762] 00000080:00200000:3.0:1789760768.449365:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.449371:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.449386:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68661248 i_blocks 134128 00000020:00200000:3.0:1789760768.449390:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.449400:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.449433:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68661248, count: 4096 00000080:00200000:3.0:1789760768.449440:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68661248, bytes: 4096 00000080:00200000:3.0:1789760768.449452:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.449461:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.449463:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68661248, 68665344) 0 0 00000080:00200000:3.0:1789760768.449468:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.449485:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16763, 16763] 00020000:00200000:3.0:1789760768.449493:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68661248 chunk: [68661248, 68665344) 68665344, 4096 00020000:00200000:3.0:1789760768.449498:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.449519:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16763, 16763] 00000008:00010000:3.0:1789760768.449538:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.449544:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.449555:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68661248, 68665344) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.449570:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68661248=0x417b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.450009:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.450011:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68665344 00000020:00010000:3.0:1789760768.450020:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.450023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.450026:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.450029:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.450032:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.450040:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.450047:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.450064:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760768.450189:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 68661248<>68665344 00010000:00010000:3.0:1789760768.539871:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1789760768.573108:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16767, 16767] 00000080:00200000:3.0:1789760768.573127:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.573134:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.573150:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68681728 i_blocks 134168 00000020:00200000:3.0:1789760768.573154:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.573165:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.573202:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68681728, count: 4096 00000080:00200000:3.0:1789760768.573209:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68681728, bytes: 4096 00000080:00200000:3.0:1789760768.573220:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.573229:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.573231:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68681728, 68685824) 0 0 00000080:00200000:3.0:1789760768.573236:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.573254:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16768, 16768] 00020000:00200000:3.0:1789760768.573263:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68681728 chunk: [68681728, 68685824) 68685824, 4096 00020000:00200000:3.0:1789760768.573267:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.573288:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16768, 16768] 00000008:00010000:3.0:1789760768.573306:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.573312:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.573324:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68681728, 68685824) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.573338:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68681728=0x4180000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.573401:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.573404:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68685824 00000020:00010000:3.0:1789760768.573413:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.573416:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.573419:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.573423:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.573425:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.573435:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.573443:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.573460:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760768.573611:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 68681728<>68685824 00000080:00200000:3.0:1789760768.590465:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16768, 16768] 00000080:00200000:3.0:1789760768.590482:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.590501:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.590516:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68685824 i_blocks 134176 00000020:00200000:3.0:1789760768.590529:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.590541:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.590578:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68685824, count: 4096 00000080:00200000:3.0:1789760768.590597:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68685824, bytes: 4096 00000080:00200000:3.0:1789760768.590608:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.590617:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.590620:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68685824, 68689920) 0 0 00000080:00200000:3.0:1789760768.590625:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.590689:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16769, 16769] 00020000:00200000:3.0:1789760768.590698:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68685824 chunk: [68685824, 68689920) 68689920, 4096 00020000:00200000:3.0:1789760768.590703:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.590724:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16769, 16769] 00000008:00010000:3.0:1789760768.590743:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.590750:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.590761:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68685824, 68689920) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.590776:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68685824=0x4181000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.590836:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.590839:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68689920 00000020:00010000:3.0:1789760768.590848:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.590851:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.590855:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.590859:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.590862:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.590870:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.590877:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.590897:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760768.591563:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 68685824<>68689920 00000100:00100000:3.0:1789760768.591826:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565375744:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.609103:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984077 gen 3 00000008:00010000:3.0:1789760768.609266:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:3.0:1789760768.609275:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.609287:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.609314:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:3.0:1789760768.609345:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565375744:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760768.655248:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16771, 16771] 00000080:00200000:3.0:1789760768.655268:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.655274:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.655288:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68698112 i_blocks 134200 00000020:00200000:3.0:1789760768.655292:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.655302:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.655337:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68698112, count: 4096 00000080:00200000:3.0:1789760768.655345:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68698112, bytes: 4096 00000080:00200000:3.0:1789760768.655356:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.655366:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.655368:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68698112, 68702208) 0 0 00000080:00200000:3.0:1789760768.655373:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.655392:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16772, 16772] 00020000:00200000:3.0:1789760768.655401:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68698112 chunk: [68698112, 68702208) 68702208, 4096 00020000:00200000:3.0:1789760768.655406:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.655426:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16772, 16772] 00000008:00010000:3.0:1789760768.655445:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.655452:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.655464:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68698112, 68702208) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.655479:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68698112=0x4184000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.655536:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.655539:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68702208 00000020:00010000:3.0:1789760768.655547:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.655551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.655556:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.655559:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.655562:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.655571:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.655578:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.668631:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16772, 16772] 00000080:00200000:3.0:1789760768.668685:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.668693:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760768.733690:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565376768:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.753626:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984085 gen 3 00000008:00010000:3.0:1789760768.753710:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760768.753718:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.753729:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.753753:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760768.753785:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565376768:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.754626:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760768.754670:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565376896:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.787868:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565377152:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760768.854159:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16783, 16783] 00000080:00200000:3.0:1789760768.854177:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760768.854183:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760768.854198:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68747264 i_blocks 134296 00000020:00200000:3.0:1789760768.854202:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760768.854214:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760768.854284:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68747264, count: 4096 00000080:00200000:3.0:1789760768.854292:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68747264, bytes: 4096 00000080:00200000:3.0:1789760768.854305:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760768.854314:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760768.854317:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68747264, 68751360) 0 0 00000080:00200000:3.0:1789760768.854321:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760768.854339:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16784, 16784] 00020000:00200000:3.0:1789760768.854349:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68747264 chunk: [68747264, 68751360) 68751360, 4096 00020000:00200000:3.0:1789760768.854354:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760768.854376:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16784, 16784] 00000008:00010000:3.0:1789760768.854395:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760768.854402:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760768.854415:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68747264, 68751360) append 0 append lockpos 0 00000080:00200000:3.0:1789760768.854428:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68747264=0x4190000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760768.854489:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760768.854492:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68751360 00000020:00010000:3.0:1789760768.854502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844242038 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760768.854506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844242038 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.854509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760768.854513:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844242038 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760768.854515:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760768.854537:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760768.854545:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760768.854565:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760768.855086:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 68747264<>68751360 00000100:00100000:3.0:1789760768.855138:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760768.855160:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565377664:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.868245:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984092 gen 3 00000008:00010000:3.0:1789760768.868297:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:3.0:1789760768.868307:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.868318:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.868342:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:3.0:1789760768.868372:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565377664:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.869410:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565377792:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.882133:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984093 gen 3 00000008:00010000:3.0:1789760768.882182:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760768.882190:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.882200:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.882225:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760768.882256:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565377792:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.883866:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760768.883892:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565377920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.897711:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984094 gen 3 00000008:00010000:3.0:1789760768.897760:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760768.897769:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.897779:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.897803:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760768.897833:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565377920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.898839:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565378048:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.916203:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984095 gen 3 00000008:00010000:3.0:1789760768.916257:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760768.916266:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760768.916278:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760768.916301:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760768.916342:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565378048:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.917157:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760768.917174:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565378176:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.938365:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984096 gen 3 00000100:00100000:3.0:1789760768.953254:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760768.953274:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565378432:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760768.966736:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984098 gen 3 00000100:00100000:3.0:1789760768.987432:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760768.987454:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565378688:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.001196:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984100 gen 3 00000008:00010000:3.0:1789760769.001327:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760769.001336:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.001347:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.001468:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760769.001507:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565378688:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.003026:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760769.003049:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565378816:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.014055:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984101 gen 3 00000008:00010000:3.0:1789760769.014108:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760769.014116:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.014126:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.014216:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760769.014248:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565378816:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760769.036594:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16794, 16794] 00000080:00200000:3.0:1789760769.036622:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760769.036628:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760769.037890:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565379072:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760769.050928:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16795, 16795] 00000080:00200000:3.0:1789760769.050945:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760769.050953:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760769.211423:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565380224:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.283419:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565380736:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.304071:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984116 gen 3 00000008:00010000:3.0:1789760769.304123:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760769.304133:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.304144:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.304170:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760769.304202:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565380736:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.305332:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565380864:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760769.372997:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16812, 16812] 00000080:00200000:3.0:1789760769.373048:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760769.373055:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760769.373071:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68866048 i_blocks 134528 00000020:00200000:3.0:1789760769.373075:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760769.373084:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760769.373151:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68866048, count: 4096 00000080:00200000:3.0:1789760769.373320:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68866048, bytes: 4096 00000080:00200000:3.0:1789760769.373339:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760769.373347:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760769.373350:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68866048, 68870144) 0 0 00000080:00200000:3.0:1789760769.373355:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760769.373372:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16813, 16813] 00020000:00200000:3.0:1789760769.373383:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68866048 chunk: [68866048, 68870144) 68870144, 4096 00020000:00200000:3.0:1789760769.373387:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760769.373410:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16813, 16813] 00000008:00010000:3.0:1789760769.373427:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760769.373434:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760769.373446:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68866048, 68870144) append 0 append lockpos 0 00000080:00200000:3.0:1789760769.373460:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68866048=0x41ad000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760769.373526:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760769.373528:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68870144 00000020:00010000:3.0:1789760769.373536:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760769.373539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760769.373543:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760769.373547:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760769.373550:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760769.373558:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760769.373631:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760769.373680:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760769.373873:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 68866048<>68870144 00000100:00100000:3.0:1789760769.373920:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760769.373941:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565381376:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.386399:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984121 gen 3 00000008:00010000:3.0:1789760769.386451:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:3.0:1789760769.386460:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.386472:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.386587:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:3.0:1789760769.386618:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565381376:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760769.408692:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16814, 16814] 00000080:00200000:3.0:1789760769.408709:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760769.408715:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760769.408730:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 68874240 i_blocks 134544 00000020:00200000:3.0:1789760769.408734:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760769.408747:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760769.408786:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 68874240, count: 4096 00000080:00200000:3.0:1789760769.408794:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 68874240, bytes: 4096 00000080:00200000:3.0:1789760769.408807:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760769.408815:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760769.408817:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [68874240, 68878336) 0 0 00000080:00200000:3.0:1789760769.408821:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760769.408838:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16815, 16815] 00020000:00200000:3.0:1789760769.408847:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 68874240 chunk: [68874240, 68878336) 68878336, 4096 00020000:00200000:3.0:1789760769.408852:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760769.408872:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16815, 16815] 00000008:00010000:3.0:1789760769.409048:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760769.409055:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760769.409067:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [68874240, 68878336) append 0 append lockpos 0 00000080:00200000:3.0:1789760769.409081:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=68874240=0x41af000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760769.409140:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760769.409143:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 68878336 00000020:00010000:3.0:1789760769.409151:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760769.409154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760769.409157:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760769.409160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760769.409163:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760769.409170:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760769.409178:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760769.409197:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760769.409425:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 68874240<>68878336 00000080:00200000:3.0:1789760769.424354:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16815, 16815] 00000080:00200000:3.0:1789760769.424374:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760769.424380:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760769.426408:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565381760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.449441:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565382016:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.457525:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984126 gen 3 00000008:00010000:3.0:1789760769.457587:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760769.457595:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.457606:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.457629:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760769.457698:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565382016:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.459728:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565382144:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.477079:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984127 gen 3 00000100:00100000:3.0:1789760769.514085:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565382528:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.523036:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984130 gen 3 00000008:00010000:3.0:1789760769.523088:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760769.523096:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.523107:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.523132:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760769.523162:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565382528:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.524221:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565382656:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.531973:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984131 gen 3 00000008:00010000:3.0:1789760769.532027:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760769.532035:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.532047:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.532075:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760769.532106:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565382656:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.533123:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760769.533144:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565382784:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.544591:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984132 gen 3 00000008:00010000:3.0:1789760769.544672:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760769.544681:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.544693:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.544773:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760769.544804:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565382784:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.545826:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565382912:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760769.562870:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1789760769.664615:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16831, 16831] 00000080:00200000:3.0:1789760769.664632:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760769.664638:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760769.665465:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565383808:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.848105:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760769.848134:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565385216:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.869290:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984151 gen 3 00000008:00010000:3.0:1789760769.869342:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760769.869351:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.869361:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.869383:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760769.869414:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565385216:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.870265:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565385344:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.894107:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984152 gen 3 00000008:00010000:3.0:1789760769.894151:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760769.894416:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.894428:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.894451:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760769.894479:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565385344:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.896636:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760769.896728:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565385472:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.915507:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984153 gen 3 00000008:00010000:3.0:1789760769.915567:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760769.915583:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.915596:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.915622:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760769.915737:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565385472:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.919473:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760769.919500:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565385600:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.934629:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984154 gen 3 00000008:00010000:3.0:1789760769.934708:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760769.934716:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.934962:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.934990:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760769.935036:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565385600:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760769.937243:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:3.0:1789760769.937874:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 69005312<>69009408 00000100:00100000:3.0:1789760769.937960:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565385728:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.957885:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984155 gen 3 00000008:00010000:3.0:1789760769.957945:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760769.957953:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.957963:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.957984:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760769.958073:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565385728:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.958834:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565385856:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.985214:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984156 gen 3 00000008:00010000:3.0:1789760769.985268:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760769.985277:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760769.985290:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760769.985318:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760769.985348:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565385856:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760769.986509:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760769.986530:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565385984:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760770.017697:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16849, 16849] 00000080:00200000:3.0:1789760770.017763:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.017770:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.017785:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69017600 i_blocks 134824 00000020:00200000:3.0:1789760770.017789:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.017798:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.017846:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69017600, count: 4096 00000080:00200000:3.0:1789760770.017856:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69017600, bytes: 4096 00000080:00200000:3.0:1789760770.017870:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.017879:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.017882:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69017600, 69021696) 0 0 00000080:00200000:3.0:1789760770.017886:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.017903:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16850, 16850] 00020000:00200000:3.0:1789760770.017912:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69017600 chunk: [69017600, 69021696) 69021696, 4096 00020000:00200000:3.0:1789760770.018086:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.018110:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16850, 16850] 00000008:00010000:3.0:1789760770.018129:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.018134:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.018145:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69017600, 69021696) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.018163:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69017600=0x41d2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.018227:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.018230:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69021696 00000020:00010000:3.0:1789760770.018239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.018242:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.018245:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.018249:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.018251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.018259:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.018267:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.037611:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16850, 16850] 00000080:00200000:3.0:1789760770.037675:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.037681:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.037696:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69021696 i_blocks 134832 00000020:00200000:3.0:1789760770.037701:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.037713:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.037751:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69021696, count: 4096 00000080:00200000:3.0:1789760770.037759:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69021696, bytes: 4096 00000080:00200000:3.0:1789760770.037769:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.037779:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.037781:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69021696, 69025792) 0 0 00000080:00200000:3.0:1789760770.037786:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.037803:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16851, 16851] 00020000:00200000:3.0:1789760770.037811:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69021696 chunk: [69021696, 69025792) 69025792, 4096 00020000:00200000:3.0:1789760770.037816:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.037835:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16851, 16851] 00000008:00010000:3.0:1789760770.037862:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.037869:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.037880:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69021696, 69025792) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.037894:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69021696=0x41d3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.037953:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.037957:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69025792 00000020:00010000:3.0:1789760770.037965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.037968:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.037971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.037975:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.037978:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.037987:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.037994:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760770.038012:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760770.038148:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 69021696<>69025792 00000080:00200000:3.0:1789760770.058109:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16851, 16851] 00000080:00200000:3.0:1789760770.058141:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.058148:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.058165:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69025792 i_blocks 134840 00000020:00200000:3.0:1789760770.058171:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.058183:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.058220:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69025792, count: 4096 00000080:00200000:3.0:1789760770.058228:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69025792, bytes: 4096 00000080:00200000:3.0:1789760770.058243:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.058253:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.058255:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69025792, 69029888) 0 0 00000080:00200000:3.0:1789760770.058260:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.058276:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16852, 16852] 00020000:00200000:3.0:1789760770.058285:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69025792 chunk: [69025792, 69029888) 69029888, 4096 00020000:00200000:3.0:1789760770.058290:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.058430:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16852, 16852] 00000008:00010000:3.0:1789760770.058450:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.058456:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.058467:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69025792, 69029888) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.058480:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69025792=0x41d4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.058759:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.058763:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69029888 00000020:00010000:3.0:1789760770.058772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.058776:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.058779:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.058783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.058786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.058796:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.058803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760770.058827:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760770.059038:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66d80 - object 0x240000400:5 offset 69025792<>69029888 00000080:00200000:3.0:1789760770.091412:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16852, 16852] 00000080:00200000:3.0:1789760770.091431:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.091438:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.091452:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69029888 i_blocks 134848 00000020:00200000:3.0:1789760770.091456:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.091466:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.091502:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69029888, count: 4096 00000080:00200000:3.0:1789760770.091520:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69029888, bytes: 4096 00000080:00200000:3.0:1789760770.091532:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.091541:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.091544:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69029888, 69033984) 0 0 00000080:00200000:3.0:1789760770.091549:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.091908:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16853, 16853] 00020000:00200000:3.0:1789760770.091988:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69029888 chunk: [69029888, 69033984) 69033984, 4096 00020000:00200000:3.0:1789760770.091994:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.092018:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16853, 16853] 00000008:00010000:3.0:1789760770.092040:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.092046:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.092058:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69029888, 69033984) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.092072:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69029888=0x41d5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.092124:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.092127:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69033984 00000020:00010000:3.0:1789760770.092135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.092139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.092142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.092145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.092149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.092157:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.092164:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:3.0:1789760770.094723:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760770.095208:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565386496:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.106543:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984161 gen 3 00000008:00010000:3.0:1789760770.106603:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:3.0:1789760770.106612:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760770.106624:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760770.106688:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:3.0:1789760770.106729:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565386496:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760770.127672:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16854, 16854] 00000080:00200000:3.0:1789760770.127689:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.127705:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.127722:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69038080 i_blocks 134864 00000020:00200000:3.0:1789760770.127726:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.127737:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.127773:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69038080, count: 4096 00000080:00200000:3.0:1789760770.127782:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69038080, bytes: 4096 00000080:00200000:3.0:1789760770.127792:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.127801:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.127803:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69038080, 69042176) 0 0 00000080:00200000:3.0:1789760770.127808:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.127826:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16855, 16855] 00020000:00200000:3.0:1789760770.127835:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69038080 chunk: [69038080, 69042176) 69042176, 4096 00020000:00200000:3.0:1789760770.127841:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.127860:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16855, 16855] 00000008:00010000:3.0:1789760770.127878:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.127884:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.127896:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69038080, 69042176) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.127910:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69038080=0x41d7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.127958:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.127961:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69042176 00000020:00010000:3.0:1789760770.127968:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.127971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.127974:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.127978:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.127986:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.127995:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.128001:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.152206:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16855, 16855] 00000080:00200000:3.0:1789760770.152227:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.152233:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.152249:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69042176 i_blocks 134872 00000020:00200000:3.0:1789760770.152254:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.152267:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200000:3.0:1789760770.168923:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16856, 16856] 00000080:00200000:3.0:1789760770.168971:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.168977:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.168993:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69046272 i_blocks 134880 00000020:00200000:3.0:1789760770.168996:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.169004:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.169051:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69046272, count: 4096 00000080:00200000:3.0:1789760770.169060:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69046272, bytes: 4096 00000080:00200000:3.0:1789760770.169072:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.169081:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.169083:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69046272, 69050368) 0 0 00000080:00200000:3.0:1789760770.169088:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.169105:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16857, 16857] 00020000:00200000:3.0:1789760770.169114:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69046272 chunk: [69046272, 69050368) 69050368, 4096 00020000:00200000:3.0:1789760770.169119:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.169139:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16857, 16857] 00000008:00010000:3.0:1789760770.169157:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.169164:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.169175:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69046272, 69050368) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.169189:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69046272=0x41d9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.169308:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.169312:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69050368 00000020:00010000:3.0:1789760770.169320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.169324:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.169327:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.169330:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.169333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.169341:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.169348:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760770.169365:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760770.169614:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 69046272<>69050368 00000080:00200000:3.0:1789760770.194364:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16857, 16857] 00000080:00200000:3.0:1789760770.194383:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.194389:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.194475:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69050368 i_blocks 134888 00000020:00200000:3.0:1789760770.194480:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.194490:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.194528:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69050368, count: 4096 00000080:00200000:3.0:1789760770.194536:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69050368, bytes: 4096 00000080:00200000:3.0:1789760770.194549:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.194559:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.194561:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69050368, 69054464) 0 0 00000080:00200000:3.0:1789760770.194565:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.194595:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16858, 16858] 00020000:00200000:3.0:1789760770.194605:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69050368 chunk: [69050368, 69054464) 69054464, 4096 00020000:00200000:3.0:1789760770.194610:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.194631:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16858, 16858] 00000008:00010000:3.0:1789760770.194814:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.195029:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.195045:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69050368, 69054464) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.195059:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69050368=0x41da000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.195127:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.195130:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69054464 00000020:00010000:3.0:1789760770.195140:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.195144:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.195147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.195151:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.195154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.195164:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.195171:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760770.195249:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760770.195535:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 69050368<>69054464 00000080:00200000:3.0:1789760770.214419:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16858, 16858] 00000080:00200000:3.0:1789760770.214435:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.214441:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.214455:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69054464 i_blocks 134896 00000020:00200000:3.0:1789760770.214458:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.214467:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.214499:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69054464, count: 4096 00000080:00200000:3.0:1789760770.214505:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69054464, bytes: 4096 00000080:00200000:3.0:1789760770.214514:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.214977:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.214980:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69054464, 69058560) 0 0 00000080:00200000:3.0:1789760770.214986:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.215008:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16859, 16859] 00020000:00200000:3.0:1789760770.215018:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69054464 chunk: [69054464, 69058560) 69058560, 4096 00020000:00200000:3.0:1789760770.215024:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.215048:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16859, 16859] 00000008:00010000:3.0:1789760770.215070:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.215077:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.215091:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69054464, 69058560) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.215105:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69054464=0x41db000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.215167:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.215170:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69058560 00000020:00010000:3.0:1789760770.215179:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.215182:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.215185:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.215188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.215191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.215200:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.215207:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760770.215229:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760770.215441:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 69054464<>69058560 00000080:00200000:3.0:1789760770.236277:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16859, 16859] 00000080:00200000:3.0:1789760770.236302:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.236309:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.236336:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69058560 i_blocks 134904 00000020:00200000:3.0:1789760770.236341:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.236356:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.236390:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69058560, count: 4096 00000080:00200000:3.0:1789760770.236399:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69058560, bytes: 4096 00000080:00200000:3.0:1789760770.236414:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.236424:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.236427:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69058560, 69062656) 0 0 00000080:00200000:3.0:1789760770.236431:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.236448:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16860, 16860] 00020000:00200000:3.0:1789760770.236458:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69058560 chunk: [69058560, 69062656) 69062656, 4096 00020000:00200000:3.0:1789760770.236462:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.236764:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16860, 16860] 00000008:00010000:3.0:1789760770.236787:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.236793:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.236808:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69058560, 69062656) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.236825:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69058560=0x41dc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.236901:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.236904:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69062656 00000020:00010000:3.0:1789760770.236913:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.236916:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.236920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.236923:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.236927:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.236937:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.236945:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760770.236979:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760770.237141:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 69058560<>69062656 00000080:00200000:3.0:1789760770.254007:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16860, 16860] 00000080:00200000:3.0:1789760770.254024:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.254031:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.254046:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69062656 i_blocks 134912 00000020:00200000:3.0:1789760770.254050:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.254059:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.254093:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69062656, count: 4096 00000080:00200000:3.0:1789760770.254101:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69062656, bytes: 4096 00000080:00200000:3.0:1789760770.254111:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.254121:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.254123:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69062656, 69066752) 0 0 00000080:00200000:3.0:1789760770.254128:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.254145:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16861, 16861] 00020000:00200000:3.0:1789760770.254218:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69062656 chunk: [69062656, 69066752) 69066752, 4096 00020000:00200000:3.0:1789760770.254223:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.254253:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16861, 16861] 00000008:00010000:3.0:1789760770.254269:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.254275:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.254287:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69062656, 69066752) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.254300:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69062656=0x41dd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.254364:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.254367:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69066752 00000020:00010000:3.0:1789760770.254375:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.254379:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.254382:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.254386:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.254389:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.254396:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.254403:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760770.254420:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760770.254549:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 69062656<>69066752 00000080:00200000:3.0:1789760770.265228:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16861, 16861] 00000080:00200000:3.0:1789760770.265293:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.265300:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.265315:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69066752 i_blocks 134920 00000020:00200000:3.0:1789760770.265319:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.265328:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.265363:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69066752, count: 4096 00000080:00200000:3.0:1789760770.265370:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69066752, bytes: 4096 00000080:00200000:3.0:1789760770.265382:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.265391:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.265394:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69066752, 69070848) 0 0 00000080:00200000:3.0:1789760770.265398:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.265417:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16862, 16862] 00020000:00200000:3.0:1789760770.265426:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69066752 chunk: [69066752, 69070848) 69070848, 4096 00020000:00200000:3.0:1789760770.265431:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.265451:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16862, 16862] 00000008:00010000:3.0:1789760770.265470:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.265476:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00000080:00200000:3.0:1789760770.265488:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69066752, 69070848) append 0 append lockpos 0 00000080:00200000:3.0:1789760770.265501:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69066752=0x41de000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760770.265561:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760770.265564:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69070848 00000020:00010000:3.0:1789760770.265582:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760770.265587:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.265590:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760770.265593:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760770.265597:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760770.265605:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760770.265612:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760770.265633:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760770.265846:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 69066752<>69070848 00000080:00200000:3.0:1789760770.280416:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16862, 16862] 00000080:00200000:3.0:1789760770.280435:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760770.280442:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760770.280459:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69070848 i_blocks 134928 00000020:00200000:3.0:1789760770.280464:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760770.280475:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760770.280523:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69070848, count: 4096 00000080:00200000:3.0:1789760770.280531:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69070848, bytes: 4096 00000080:00200000:3.0:1789760770.280542:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760770.280553:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760770.280555:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69070848, 69074944) 0 0 00000080:00200000:3.0:1789760770.280560:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760770.280586:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16863, 16863] 00020000:00200000:3.0:1789760770.280595:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69070848 chunk: [69070848, 69074944) 69074944, 4096 00020000:00200000:3.0:1789760770.280600:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760770.280621:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16863, 16863] 00000008:00010000:3.0:1789760770.280639:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760770.281216:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e430, osc lock ffff93f8457e7b50, flags 0x0 00010000:00010000:3.0:1789760770.590676:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:1.0:1789760770.659008:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.659071:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16885, 16885] 00020000:00200000:1.0:1789760770.659079:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69160960 chunk: [69160960, 69165056) 69165056, 4096 00020000:00200000:1.0:1789760770.659084:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.659102:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16885, 16885] 00000008:00010000:1.0:1789760770.659379:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.659386:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.659398:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69160960, 69165056) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.659412:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69160960=0x41f5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.659531:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.659534:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69165056 00000020:00010000:1.0:1789760770.659543:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.659546:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.659609:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.659614:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.659617:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.659626:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.659634:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.659801:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760770.659932:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 69160960<>69165056 00000100:00100000:3.0:1789760770.671179:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565390720:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.679328:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984194 gen 3 00000008:00010000:3.0:1789760770.679380:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760770.679390:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760770.679401:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760770.679493:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760770.679532:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565390720:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.680449:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565390848:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.689169:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984195 gen 3 00000008:00010000:3.0:1789760770.689217:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760770.689226:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760770.689248:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760770.689269:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760770.689301:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565390848:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.690257:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760770.690276:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565390976:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.700935:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984196 gen 3 00000008:00010000:3.0:1789760770.700992:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760770.701001:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760770.701012:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760770.701036:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760770.701067:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565390976:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.703201:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565391104:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.712042:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984197 gen 3 00000008:00010000:3.0:1789760770.712096:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760770.712104:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760770.712115:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760770.712138:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760770.712169:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565391104:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760770.712279:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16889, 16889] 00000080:00200000:1.0:1789760770.712296:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.712302:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.712318:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69181440 i_blocks 135144 00000020:00200000:1.0:1789760770.712332:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.712343:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.712379:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69181440, count: 4096 00000080:00200000:1.0:1789760770.712386:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69181440, bytes: 4096 00000080:00200000:1.0:1789760770.712397:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.712406:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.712408:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69181440, 69185536) 0 0 00000080:00200000:1.0:1789760770.712413:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.712432:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16890, 16890] 00020000:00200000:1.0:1789760770.712440:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69181440 chunk: [69181440, 69185536) 69185536, 4096 00020000:00200000:1.0:1789760770.712445:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.712472:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16890, 16890] 00000008:00010000:1.0:1789760770.712489:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.712495:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.712506:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69181440, 69185536) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.712520:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69181440=0x41fa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.712588:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.712591:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69185536 00000020:00010000:1.0:1789760770.712599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.712602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.712605:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.712608:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.712611:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.712619:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.712755:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.712772:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760770.713364:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 69181440<>69185536 00000100:00100000:1.0:1789760770.713453:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565391232:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.726553:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760770.726595:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565391360:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.740824:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984199 gen 3 00000008:00010000:3.0:1789760770.740876:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760770.740885:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760770.740896:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760770.740925:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760770.740956:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565391360:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.742437:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565391488:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760770.759287:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69193728 i_blocks 135168 00000020:00200000:1.0:1789760770.759292:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.759304:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.759338:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69193728, count: 4096 00000080:00200000:1.0:1789760770.759345:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69193728, bytes: 4096 00000080:00200000:1.0:1789760770.759357:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.759366:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.759369:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69193728, 69197824) 0 0 00000080:00200000:1.0:1789760770.759374:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.759442:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16893, 16893] 00020000:00200000:1.0:1789760770.759452:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69193728 chunk: [69193728, 69197824) 69197824, 4096 00020000:00200000:1.0:1789760770.759457:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.759481:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16893, 16893] 00000008:00010000:1.0:1789760770.759501:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.759575:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.759589:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69193728, 69197824) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.759604:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69193728=0x41fd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.759803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.759807:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69197824 00000020:00010000:1.0:1789760770.759815:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.759866:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.759869:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.759873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.759875:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.759884:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.759892:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.759915:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760770.760164:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 69193728<>69197824 00000080:00200000:1.0:1789760770.781317:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16893, 16893] 00000080:00200000:1.0:1789760770.781337:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.781343:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.781360:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69197824 i_blocks 135176 00000020:00200000:1.0:1789760770.781363:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.781373:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.781409:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69197824, count: 4096 00000080:00200000:1.0:1789760770.781418:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69197824, bytes: 4096 00000080:00200000:1.0:1789760770.781429:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.781439:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.781442:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69197824, 69201920) 0 0 00000080:00200000:1.0:1789760770.781448:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.781467:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16894, 16894] 00020000:00200000:1.0:1789760770.781476:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69197824 chunk: [69197824, 69201920) 69201920, 4096 00020000:00200000:1.0:1789760770.781481:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.781512:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16894, 16894] 00000008:00010000:1.0:1789760770.781531:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.781538:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.781551:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69197824, 69201920) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.781575:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69197824=0x41fe000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.786350:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.786363:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69201920 00000020:00010000:1.0:1789760770.786382:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.786387:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.786392:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.786396:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.786399:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.786414:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.786425:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.786477:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760770.786876:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 69197824<>69201920 00000080:00200000:1.0:1789760770.805768:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16894, 16894] 00000080:00200000:1.0:1789760770.805846:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.805853:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.805871:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69201920 i_blocks 135184 00000020:00200000:1.0:1789760770.805876:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.805887:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.805925:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69201920, count: 4096 00000080:00200000:1.0:1789760770.805932:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69201920, bytes: 4096 00000080:00200000:1.0:1789760770.805947:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.805956:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.805958:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69201920, 69206016) 0 0 00000080:00200000:1.0:1789760770.805963:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.805981:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16895, 16895] 00020000:00200000:1.0:1789760770.805990:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69201920 chunk: [69201920, 69206016) 69206016, 4096 00020000:00200000:1.0:1789760770.806135:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.806158:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16895, 16895] 00000008:00010000:1.0:1789760770.806177:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.806182:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.806195:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69201920, 69206016) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.806210:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69201920=0x41ff000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.806277:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.806280:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69206016 00000020:00010000:1.0:1789760770.806288:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.806291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.806295:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.806299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.806302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.806310:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.806317:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.806339:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760770.806470:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 69201920<>69206016 00000080:00200000:1.0:1789760770.828951:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16895, 16895] 00000080:00200000:1.0:1789760770.828967:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.828974:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.828992:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69206016 i_blocks 135192 00000020:00200000:1.0:1789760770.829156:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.829167:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.829207:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69206016, count: 4096 00000080:00200000:1.0:1789760770.829216:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69206016, bytes: 4096 00000080:00200000:1.0:1789760770.829229:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.829238:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.829241:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69206016, 69210112) 0 0 00000080:00200000:1.0:1789760770.829246:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.829264:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16896, 16896] 00020000:00200000:1.0:1789760770.829275:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69206016 chunk: [69206016, 69210112) 69210112, 4096 00020000:00200000:1.0:1789760770.829280:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.829301:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16896, 16896] 00000008:00010000:1.0:1789760770.829318:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.829325:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.829337:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69206016, 69210112) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.829351:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69206016=0x4200000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.829411:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.829415:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69210112 00000020:00010000:1.0:1789760770.829423:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.829427:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.829430:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.829434:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.829436:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.829444:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.829452:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.829468:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760770.830726:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 69206016<>69210112 00000080:00200000:1.0:1789760770.846349:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16896, 16896] 00000080:00200000:1.0:1789760770.846368:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.846374:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.846390:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69210112 i_blocks 135200 00000020:00200000:1.0:1789760770.846394:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.846405:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.846439:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69210112, count: 4096 00000080:00200000:1.0:1789760770.846447:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69210112, bytes: 4096 00000080:00200000:1.0:1789760770.846460:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.846469:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.846471:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69210112, 69214208) 0 0 00000080:00200000:1.0:1789760770.846476:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.846494:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16897, 16897] 00020000:00200000:1.0:1789760770.846506:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69210112 chunk: [69210112, 69214208) 69214208, 4096 00020000:00200000:1.0:1789760770.846510:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.846533:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16897, 16897] 00000008:00010000:1.0:1789760770.846552:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.846558:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.846579:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69210112, 69214208) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.846594:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69210112=0x4201000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.846726:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.846729:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69214208 00000020:00010000:1.0:1789760770.846738:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.846741:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.846744:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.846748:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.846751:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.846759:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.846765:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.846785:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760770.847048:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 69210112<>69214208 00000080:00200000:1.0:1789760770.860079:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16897, 16897] 00000080:00200000:1.0:1789760770.860096:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.860106:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.860122:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69214208 i_blocks 135208 00000020:00200000:1.0:1789760770.860125:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.860135:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.860169:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69214208, count: 4096 00000080:00200000:1.0:1789760770.860177:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69214208, bytes: 4096 00000080:00200000:1.0:1789760770.860188:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.860196:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.860199:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69214208, 69218304) 0 0 00000080:00200000:1.0:1789760770.860205:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.860223:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16898, 16898] 00020000:00200000:1.0:1789760770.860233:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69214208 chunk: [69214208, 69218304) 69218304, 4096 00020000:00200000:1.0:1789760770.860238:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.860259:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16898, 16898] 00000008:00010000:1.0:1789760770.860278:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.860284:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.860295:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69214208, 69218304) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.860310:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69214208=0x4202000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.860372:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.860375:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69218304 00000020:00010000:1.0:1789760770.860384:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.860387:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.860390:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.860394:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.860396:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.860403:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.860409:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.860426:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760770.860744:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 69214208<>69218304 00000080:00200000:1.0:1789760770.875144:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16898, 16898] 00000080:00200000:1.0:1789760770.875160:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.875167:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.875182:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69218304 i_blocks 135216 00000020:00200000:1.0:1789760770.875186:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.875197:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.875235:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69218304, count: 4096 00000080:00200000:1.0:1789760770.875242:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69218304, bytes: 4096 00000080:00200000:1.0:1789760770.875252:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.875260:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.875263:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69218304, 69222400) 0 0 00000080:00200000:1.0:1789760770.875267:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.875353:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16899, 16899] 00020000:00200000:1.0:1789760770.875363:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69218304 chunk: [69218304, 69222400) 69222400, 4096 00020000:00200000:1.0:1789760770.875368:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.875388:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16899, 16899] 00000008:00010000:1.0:1789760770.875407:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.875412:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.875424:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69218304, 69222400) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.875438:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69218304=0x4203000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.875513:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.875516:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69222400 00000020:00010000:1.0:1789760770.875720:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.875726:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.875730:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.875733:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.875736:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.875745:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.875752:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.875771:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760770.876174:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 69218304<>69222400 00000080:00200000:1.0:1789760770.889764:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16899, 16899] 00000080:00200000:1.0:1789760770.889793:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.889800:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.889816:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69222400 i_blocks 135224 00000020:00200000:1.0:1789760770.889820:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.889830:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.889867:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69222400, count: 4096 00000080:00200000:1.0:1789760770.889874:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69222400, bytes: 4096 00000080:00200000:1.0:1789760770.889884:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.889893:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.889895:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69222400, 69226496) 0 0 00000080:00200000:1.0:1789760770.889900:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.889917:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16900, 16900] 00020000:00200000:1.0:1789760770.889927:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69222400 chunk: [69222400, 69226496) 69226496, 4096 00020000:00200000:1.0:1789760770.889933:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.889953:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16900, 16900] 00000008:00010000:1.0:1789760770.889972:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.889978:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.889990:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69222400, 69226496) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.890005:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69222400=0x4204000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.890066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.890069:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69226496 00000020:00010000:1.0:1789760770.890077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.890081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.890084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.890088:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.890091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.890099:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.890106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.890125:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760770.890269:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 69222400<>69226496 00000080:00200000:1.0:1789760770.902811:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16900, 16900] 00000080:00200000:1.0:1789760770.902828:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.902834:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.902858:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69226496 i_blocks 135232 00000020:00200000:1.0:1789760770.902863:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.902872:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.902907:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69226496, count: 4096 00000080:00200000:1.0:1789760770.902914:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69226496, bytes: 4096 00000080:00200000:1.0:1789760770.902924:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.902934:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.902936:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69226496, 69230592) 0 0 00000080:00200000:1.0:1789760770.902941:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.902958:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16901, 16901] 00020000:00200000:1.0:1789760770.902966:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69226496 chunk: [69226496, 69230592) 69230592, 4096 00020000:00200000:1.0:1789760770.902970:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.902986:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16901, 16901] 00000008:00010000:1.0:1789760770.902999:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.903004:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.903014:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69226496, 69230592) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.903025:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69226496=0x4205000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.903082:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.903084:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69230592 00000020:00010000:1.0:1789760770.903090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.903093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.903096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.903100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.903103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.903110:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.903116:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.903142:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760770.903296:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 69226496<>69230592 00000080:00200000:1.0:1789760770.915573:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16901, 16901] 00000080:00200000:1.0:1789760770.915592:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.915599:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.915616:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69230592 i_blocks 135240 00000020:00200000:1.0:1789760770.915620:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.915631:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.915685:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69230592, count: 4096 00000080:00200000:1.0:1789760770.915694:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69230592, bytes: 4096 00000080:00200000:1.0:1789760770.915705:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.915714:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.915717:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69230592, 69234688) 0 0 00000080:00200000:1.0:1789760770.915722:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.915739:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16902, 16902] 00020000:00200000:1.0:1789760770.915749:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69230592 chunk: [69230592, 69234688) 69234688, 4096 00020000:00200000:1.0:1789760770.915755:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.915776:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16902, 16902] 00000008:00010000:1.0:1789760770.915797:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.915804:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.915817:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69230592, 69234688) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.915837:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69230592=0x4206000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.915896:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.915898:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69234688 00000020:00010000:1.0:1789760770.915906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.915908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.915910:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.915913:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.915915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.915920:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.915925:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.915937:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760770.916062:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 69230592<>69234688 00000080:00200000:1.0:1789760770.927202:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16902, 16902] 00000080:00200000:1.0:1789760770.927218:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.927224:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.927238:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69234688 i_blocks 135248 00000020:00200000:1.0:1789760770.927242:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.927251:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.927286:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69234688, count: 4096 00000080:00200000:1.0:1789760770.927294:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69234688, bytes: 4096 00000080:00200000:1.0:1789760770.927304:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.927313:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.927315:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69234688, 69238784) 0 0 00000080:00200000:1.0:1789760770.927320:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.927335:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16903, 16903] 00020000:00200000:1.0:1789760770.927345:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69234688 chunk: [69234688, 69238784) 69238784, 4096 00020000:00200000:1.0:1789760770.927350:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.927372:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16903, 16903] 00000008:00010000:1.0:1789760770.927391:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.927397:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.927408:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69234688, 69238784) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.927422:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69234688=0x4207000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.927481:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.927485:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69238784 00000020:00010000:1.0:1789760770.927493:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.927497:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.927500:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.927504:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.927507:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.927516:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.927523:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.927543:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760770.928036:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 69234688<>69238784 00000080:00200000:1.0:1789760770.939209:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16903, 16903] 00000080:00200000:1.0:1789760770.939225:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760770.939230:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760770.939246:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69238784 i_blocks 135256 00000020:00200000:1.0:1789760770.939250:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760770.939260:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760770.939291:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69238784, count: 4096 00000080:00200000:1.0:1789760770.939299:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69238784, bytes: 4096 00000080:00200000:1.0:1789760770.939308:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760770.939317:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760770.939319:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69238784, 69242880) 0 0 00000080:00200000:1.0:1789760770.939324:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760770.939342:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16904, 16904] 00020000:00200000:1.0:1789760770.939351:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69238784 chunk: [69238784, 69242880) 69242880, 4096 00020000:00200000:1.0:1789760770.939355:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760770.939375:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16904, 16904] 00000008:00010000:1.0:1789760770.939392:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760770.939398:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760770.939408:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69238784, 69242880) append 0 append lockpos 0 00000080:00200000:1.0:1789760770.939421:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69238784=0x4208000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760770.939478:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760770.939481:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69242880 00000020:00010000:1.0:1789760770.939489:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760770.939492:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.939496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760770.939499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760770.939502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760770.939509:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760770.939515:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760770.939531:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760770.940809:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 69238784<>69242880 00000100:00100000:1.0:1789760770.940877:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565393024:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.958805:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565393152:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.974026:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984213 gen 3 00000008:00010000:3.0:1789760770.974075:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760770.974083:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760770.975719:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760770.975756:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760770.975795:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565393152:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760770.978252:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565393280:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.003315:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984214 gen 3 00000100:00100000:1.0:1789760771.058467:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565393664:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.071860:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565393792:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.081990:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984218 gen 3 00000008:00010000:3.0:1789760771.082035:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760771.082043:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.082052:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.082075:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760771.082102:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565393792:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.083092:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760771.083111:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565393920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.097486:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984219 gen 3 00000008:00010000:3.0:1789760771.097544:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760771.097552:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.097570:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.097590:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760771.097617:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565393920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.099767:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394048:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.110602:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984220 gen 3 00000008:00010000:3.0:1789760771.110680:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760771.110690:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.110700:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.110735:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760771.110765:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394048:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.111693:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760771.111738:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394176:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.129914:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984221 gen 3 00000008:00010000:3.0:1789760771.129969:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760771.129978:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.129990:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.130016:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760771.130049:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394176:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.133614:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394304:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.146308:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984222 gen 3 00000008:00010000:3.0:1789760771.146364:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760771.146374:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.146385:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.146413:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760771.146444:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394304:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.148437:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760771.148464:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394432:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.165867:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984223 gen 3 00000008:00010000:3.0:1789760771.165930:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760771.166052:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.166066:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.166088:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760771.166123:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394432:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.167082:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394560:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.177633:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984224 gen 3 00000008:00010000:3.0:1789760771.177731:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760771.177739:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.177749:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.177772:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760771.177801:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394560:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.178771:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760771.178791:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394688:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.186266:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984225 gen 3 00000008:00010000:3.0:1789760771.186317:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760771.186326:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.186338:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.186359:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760771.186390:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394688:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.188205:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394816:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.196687:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984226 gen 3 00000008:00010000:3.0:1789760771.196733:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760771.196741:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.196752:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.196771:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760771.196798:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394816:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.199207:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760771.199230:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565394944:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.213293:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984227 gen 3 00000100:00100000:1.0:1789760771.216536:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565395072:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.226534:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565395200:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760771.263029:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16922, 16922] 00000080:00200000:1.0:1789760771.263048:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.263055:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.263071:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69316608 i_blocks 135408 00000020:00200000:1.0:1789760771.263075:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.263087:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.263121:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69316608, count: 4096 00000080:00200000:1.0:1789760771.263130:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69316608, bytes: 4096 00000080:00200000:1.0:1789760771.263140:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.263149:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.263152:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69316608, 69320704) 0 0 00000080:00200000:1.0:1789760771.263157:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.263173:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16923, 16923] 00020000:00200000:1.0:1789760771.263183:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69316608 chunk: [69316608, 69320704) 69320704, 4096 00020000:00200000:1.0:1789760771.263187:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.263209:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16923, 16923] 00000008:00010000:1.0:1789760771.263229:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.263235:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.263248:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69316608, 69320704) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.263262:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69316608=0x421b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.263322:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.263325:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69320704 00000020:00010000:1.0:1789760771.263333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.263336:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.263340:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.263343:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.263346:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.263353:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.263360:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.263380:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760771.263768:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 69316608<>69320704 00000100:00100000:1.0:1789760771.263817:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565395456:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.277428:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565395584:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.301074:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565395840:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760771.313479:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760771.313510:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565395968:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760771.381002:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16929, 16929] 00000080:00200000:3.0:1789760771.381018:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760771.381024:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760771.381040:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69345280 i_blocks 135464 00000020:00200000:3.0:1789760771.381043:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760771.381053:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760771.381084:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69345280, count: 4096 00000080:00200000:3.0:1789760771.381091:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69345280, bytes: 4096 00000080:00200000:3.0:1789760771.381116:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760771.381125:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760771.381128:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69345280, 69349376) 0 0 00000080:00200000:3.0:1789760771.381139:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760771.381155:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16930, 16930] 00020000:00200000:3.0:1789760771.381165:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69345280 chunk: [69345280, 69349376) 69349376, 4096 00020000:00200000:3.0:1789760771.381170:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760771.381189:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16930, 16930] 00000008:00010000:3.0:1789760771.381207:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760771.381214:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760771.381226:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69345280, 69349376) append 0 append lockpos 0 00000080:00200000:3.0:1789760771.381240:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69345280=0x4222000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760771.381294:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760771.381296:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69349376 00000020:00010000:3.0:1789760771.381304:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760771.381308:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760771.381311:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760771.381315:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760771.381318:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760771.381327:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760771.381334:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760771.381349:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760771.381467:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 69345280<>69349376 00000100:00100000:3.0:1789760771.381724:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760771.381743:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565396352:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.399195:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984238 gen 3 00000008:00010000:3.0:1789760771.399249:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:3.0:1789760771.399258:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.399270:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.399300:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:3.0:1789760771.399331:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565396352:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760771.400785:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565396480:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760771.417936:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16931, 16931] 00000080:00200000:3.0:1789760771.417963:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760771.417969:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760771.417987:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69353472 i_blocks 135480 00000020:00200000:3.0:1789760771.417993:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760771.418011:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760771.418046:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69353472, count: 4096 00000080:00200000:3.0:1789760771.418054:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69353472, bytes: 4096 00000080:00200000:3.0:1789760771.418069:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760771.418080:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760771.418082:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69353472, 69357568) 0 0 00000080:00200000:3.0:1789760771.418088:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760771.418105:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16932, 16932] 00020000:00200000:3.0:1789760771.418116:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69353472 chunk: [69353472, 69357568) 69357568, 4096 00020000:00200000:3.0:1789760771.418122:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760771.418145:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16932, 16932] 00000008:00010000:3.0:1789760771.418165:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760771.418172:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760771.418185:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69353472, 69357568) append 0 append lockpos 0 00000080:00200000:3.0:1789760771.418200:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69353472=0x4224000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760771.418266:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760771.418268:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69357568 00000020:00010000:3.0:1789760771.418276:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760771.418280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760771.418283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760771.418287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760771.418290:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760771.418299:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760771.418306:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760771.418327:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760771.418463:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 69353472<>69357568 00000080:00200000:1.0:1789760771.471151:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16933, 16933] 00000080:00200000:1.0:1789760771.471167:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.471173:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.471189:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69361664 i_blocks 135496 00000020:00200000:1.0:1789760771.471193:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.471204:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.471235:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69361664, count: 4096 00000080:00200000:1.0:1789760771.471243:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69361664, bytes: 4096 00000080:00200000:1.0:1789760771.471253:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.471262:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.471264:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69361664, 69365760) 0 0 00000080:00200000:1.0:1789760771.471270:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.471286:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16934, 16934] 00020000:00200000:1.0:1789760771.471296:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69361664 chunk: [69361664, 69365760) 69365760, 4096 00020000:00200000:1.0:1789760771.471300:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.471320:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16934, 16934] 00000008:00010000:1.0:1789760771.471339:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.471345:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.471357:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69361664, 69365760) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.471371:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69361664=0x4226000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.471428:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.471430:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69365760 00000020:00010000:1.0:1789760771.471439:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.471443:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.471447:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.471450:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.471454:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.471461:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.471468:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.471483:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760771.471620:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 69361664<>69365760 00000100:00100000:1.0:1789760771.471690:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760771.471706:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565396864:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760771.482988:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16934, 16934] 00000080:00200000:1.0:1789760771.483004:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.483010:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.483025:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69365760 i_blocks 135504 00000020:00200000:1.0:1789760771.483029:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.483039:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.483073:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69365760, count: 4096 00000080:00200000:1.0:1789760771.483080:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69365760, bytes: 4096 00000080:00200000:1.0:1789760771.483089:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.483097:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.483100:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69365760, 69369856) 0 0 00000080:00200000:1.0:1789760771.483105:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.483121:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16935, 16935] 00020000:00200000:1.0:1789760771.483130:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69365760 chunk: [69365760, 69369856) 69369856, 4096 00020000:00200000:1.0:1789760771.483134:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.483151:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16935, 16935] 00000008:00010000:1.0:1789760771.483167:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.483173:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.483184:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69365760, 69369856) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.483196:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69365760=0x4227000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.483252:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.483254:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69369856 00000020:00010000:1.0:1789760771.483263:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.483267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.483270:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.483273:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.483276:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.483284:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.483290:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.483306:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760771.483420:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 69365760<>69369856 00000080:00200000:1.0:1789760771.495623:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16935, 16935] 00000080:00200000:1.0:1789760771.495640:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.495749:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.495766:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69369856 i_blocks 135512 00000020:00200000:1.0:1789760771.495769:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.495779:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.495813:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69369856, count: 4096 00000080:00200000:1.0:1789760771.495821:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69369856, bytes: 4096 00000080:00200000:1.0:1789760771.495831:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.495839:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.495842:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69369856, 69373952) 0 0 00000080:00200000:1.0:1789760771.495847:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.495863:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16936, 16936] 00020000:00200000:1.0:1789760771.495871:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69369856 chunk: [69369856, 69373952) 69373952, 4096 00020000:00200000:1.0:1789760771.495876:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.495896:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16936, 16936] 00000008:00010000:1.0:1789760771.495914:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.495920:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.495932:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69369856, 69373952) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.495946:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69369856=0x4228000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.496285:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.496289:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69373952 00000020:00010000:1.0:1789760771.496298:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.496302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.496305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.496309:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.496312:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.496319:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.496326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.496346:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760771.496464:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 69369856<>69373952 00000080:00200000:1.0:1789760771.509048:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16936, 16936] 00000080:00200000:1.0:1789760771.509065:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.509071:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.509087:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69373952 i_blocks 135520 00000020:00200000:1.0:1789760771.509091:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.509101:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.509137:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69373952, count: 4096 00000080:00200000:1.0:1789760771.509145:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69373952, bytes: 4096 00000080:00200000:1.0:1789760771.509156:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.509165:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.509167:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69373952, 69378048) 0 0 00000080:00200000:1.0:1789760771.509172:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.509190:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16937, 16937] 00020000:00200000:1.0:1789760771.509199:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69373952 chunk: [69373952, 69378048) 69378048, 4096 00020000:00200000:1.0:1789760771.509204:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.509224:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16937, 16937] 00000008:00010000:1.0:1789760771.509244:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.509250:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.509262:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69373952, 69378048) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.509277:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69373952=0x4229000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.509340:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.509343:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69378048 00000020:00010000:1.0:1789760771.509351:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.509355:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.509358:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.509361:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.509364:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.509372:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.509378:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.509396:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760771.509519:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 69373952<>69378048 00000080:00200000:1.0:1789760771.526255:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16937, 16937] 00000080:00200000:1.0:1789760771.526272:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.526278:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.526293:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69378048 i_blocks 135528 00000020:00200000:1.0:1789760771.526297:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.526307:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.526343:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69378048, count: 4096 00000080:00200000:1.0:1789760771.526350:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69378048, bytes: 4096 00000080:00200000:1.0:1789760771.526361:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.526370:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.526372:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69378048, 69382144) 0 0 00000080:00200000:1.0:1789760771.526377:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.526394:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16938, 16938] 00020000:00200000:1.0:1789760771.526403:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69378048 chunk: [69378048, 69382144) 69382144, 4096 00020000:00200000:1.0:1789760771.526407:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.526428:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16938, 16938] 00000008:00010000:1.0:1789760771.526447:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.526454:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.526465:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69378048, 69382144) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.526479:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69378048=0x422a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.526753:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.526757:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69382144 00000020:00010000:1.0:1789760771.526765:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.526769:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.526772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.526775:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.526778:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.526785:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.526792:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.526810:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760771.526999:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 69378048<>69382144 00000080:00200000:1.0:1789760771.535975:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16938, 16938] 00000080:00200000:1.0:1789760771.535991:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.535997:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.536012:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69382144 i_blocks 135536 00000020:00200000:1.0:1789760771.536015:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.536024:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.536057:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69382144, count: 4096 00000080:00200000:1.0:1789760771.536064:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69382144, bytes: 4096 00000080:00200000:1.0:1789760771.536073:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.536082:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.536084:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69382144, 69386240) 0 0 00000080:00200000:1.0:1789760771.536088:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.536104:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16939, 16939] 00020000:00200000:1.0:1789760771.536112:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69382144 chunk: [69382144, 69386240) 69386240, 4096 00020000:00200000:1.0:1789760771.536116:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.536134:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16939, 16939] 00000008:00010000:1.0:1789760771.536150:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.536156:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.536167:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69382144, 69386240) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.536183:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69382144=0x422b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.536239:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.536241:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69386240 00000020:00010000:1.0:1789760771.536249:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.536252:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.536255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.536258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.536261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.536267:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.536274:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.536291:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760771.536431:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 69382144<>69386240 00000080:00200000:1.0:1789760771.547430:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16939, 16939] 00000080:00200000:1.0:1789760771.547450:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.547456:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.547472:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69386240 i_blocks 135544 00000020:00200000:1.0:1789760771.547478:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.547489:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.547533:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69386240, count: 4096 00000080:00200000:1.0:1789760771.547551:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69386240, bytes: 4096 00000080:00200000:1.0:1789760771.547570:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.547580:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.547582:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69386240, 69390336) 0 0 00000080:00200000:1.0:1789760771.547587:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.547604:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16940, 16940] 00020000:00200000:1.0:1789760771.547613:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69386240 chunk: [69386240, 69390336) 69390336, 4096 00020000:00200000:1.0:1789760771.547617:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.547639:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16940, 16940] 00000008:00010000:1.0:1789760771.547674:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.547681:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.547702:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69386240, 69390336) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.547717:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69386240=0x422c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.547779:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.547782:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69390336 00000020:00010000:1.0:1789760771.547790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.547794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.547797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.547800:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.547803:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.547811:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.547818:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.547839:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760771.547958:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 69386240<>69390336 00000080:00200000:1.0:1789760771.559326:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16940, 16940] 00000080:00200000:1.0:1789760771.559344:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.559351:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.559366:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69390336 i_blocks 135552 00000020:00200000:1.0:1789760771.559370:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.559381:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.559416:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69390336, count: 4096 00000080:00200000:1.0:1789760771.559423:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69390336, bytes: 4096 00000080:00200000:1.0:1789760771.559434:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.559443:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.559445:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69390336, 69394432) 0 0 00000080:00200000:1.0:1789760771.559450:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.559466:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16941, 16941] 00020000:00200000:1.0:1789760771.559475:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69390336 chunk: [69390336, 69394432) 69394432, 4096 00020000:00200000:1.0:1789760771.559479:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.559571:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16941, 16941] 00000008:00010000:1.0:1789760771.559591:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.559598:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.559610:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69390336, 69394432) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.560361:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69390336=0x422d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.560436:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.560439:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69394432 00000020:00010000:1.0:1789760771.560448:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.560452:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.560456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.560459:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.560462:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.560470:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.560478:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.560512:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760771.560823:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 69390336<>69394432 00000080:00200000:1.0:1789760771.570492:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16941, 16941] 00000080:00200000:1.0:1789760771.570509:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.570588:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.570603:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69394432 i_blocks 135560 00000020:00200000:1.0:1789760771.570608:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.570617:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.571067:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69394432, count: 4096 00000080:00200000:1.0:1789760771.571077:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69394432, bytes: 4096 00000080:00200000:1.0:1789760771.571088:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.571098:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.571100:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69394432, 69398528) 0 0 00000080:00200000:1.0:1789760771.571105:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.571122:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16942, 16942] 00020000:00200000:1.0:1789760771.571231:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69394432 chunk: [69394432, 69398528) 69398528, 4096 00020000:00200000:1.0:1789760771.571876:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.571904:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16942, 16942] 00000008:00010000:1.0:1789760771.571922:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.571928:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.571941:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69394432, 69398528) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.571956:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69394432=0x422e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.572022:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.572025:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69398528 00000020:00010000:1.0:1789760771.572033:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.572037:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.572040:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.572044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.572047:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.572056:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.572063:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.572085:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760771.572366:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 69394432<>69398528 00000080:00200000:1.0:1789760771.593907:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16942, 16942] 00000080:00200000:1.0:1789760771.593924:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.593930:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.593946:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69398528 i_blocks 135568 00000020:00200000:1.0:1789760771.593950:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.593959:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.593991:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69398528, count: 4096 00000080:00200000:1.0:1789760771.593999:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69398528, bytes: 4096 00000080:00200000:1.0:1789760771.594008:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.594016:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.594019:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69398528, 69402624) 0 0 00000080:00200000:1.0:1789760771.594023:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.594040:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16943, 16943] 00020000:00200000:1.0:1789760771.594049:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69398528 chunk: [69398528, 69402624) 69402624, 4096 00020000:00200000:1.0:1789760771.594053:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.594072:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16943, 16943] 00000008:00010000:1.0:1789760771.594087:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.594093:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.594104:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69398528, 69402624) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.594117:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69398528=0x422f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.594177:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.594180:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69402624 00000020:00010000:1.0:1789760771.594188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.594192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.594195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.594198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.594201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.594207:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.594213:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.594230:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760771.594355:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 69398528<>69402624 00010000:00010000:3.0:1789760771.610900:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:1.0:1789760771.611476:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16943, 16943] 00000080:00200000:1.0:1789760771.611492:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.611498:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.611514:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69402624 i_blocks 135576 00000020:00200000:1.0:1789760771.611518:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.611527:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.611576:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69402624, count: 4096 00000080:00200000:1.0:1789760771.611584:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69402624, bytes: 4096 00000080:00200000:1.0:1789760771.611594:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.611603:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.611606:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69402624, 69406720) 0 0 00000080:00200000:1.0:1789760771.611611:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.611629:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16944, 16944] 00020000:00200000:1.0:1789760771.611638:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69402624 chunk: [69402624, 69406720) 69406720, 4096 00020000:00200000:1.0:1789760771.611687:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.611728:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16944, 16944] 00000008:00010000:1.0:1789760771.611747:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.611753:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.611766:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69402624, 69406720) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.611788:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69402624=0x4230000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.611844:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.611847:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69406720 00000020:00010000:1.0:1789760771.611855:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.611858:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.611862:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.611865:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.611868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.611876:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.611882:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.611896:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760771.612023:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 69402624<>69406720 00000080:00200000:1.0:1789760771.625172:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16944, 16944] 00000080:00200000:1.0:1789760771.625196:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.625203:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.625232:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69406720 i_blocks 135584 00000020:00200000:1.0:1789760771.625243:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.625258:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.625291:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69406720, count: 4096 00000080:00200000:1.0:1789760771.625299:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69406720, bytes: 4096 00000080:00200000:1.0:1789760771.625315:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.625335:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.625337:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69406720, 69410816) 0 0 00000080:00200000:1.0:1789760771.625342:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.625369:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16945, 16945] 00020000:00200000:1.0:1789760771.625379:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69406720 chunk: [69406720, 69410816) 69410816, 4096 00020000:00200000:1.0:1789760771.625384:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.625406:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16945, 16945] 00000008:00010000:1.0:1789760771.625425:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.625431:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.625443:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69406720, 69410816) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.625458:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69406720=0x4231000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.625525:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.625528:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69410816 00000020:00010000:1.0:1789760771.625536:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.625540:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.625543:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0430 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.625546:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f844188a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.625549:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0430 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.625556:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.625573:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.625596:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760771.625770:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 69406720<>69410816 00000080:00200000:3.0:1789760771.645873:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16945, 16945] 00000080:00200000:3.0:1789760771.645891:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760771.645898:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760771.647120:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565398784:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760771.662141:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16946, 16946] 00000080:00200000:1.0:1789760771.662157:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.662162:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.662177:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69414912 i_blocks 135600 00000020:00200000:1.0:1789760771.662181:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.662190:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.662234:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69414912, count: 4096 00000080:00200000:1.0:1789760771.662242:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69414912, bytes: 4096 00000080:00200000:1.0:1789760771.662252:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.662260:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.662263:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69414912, 69419008) 0 0 00000080:00200000:1.0:1789760771.662267:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.662282:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16947, 16947] 00020000:00200000:1.0:1789760771.662290:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69414912 chunk: [69414912, 69419008) 69419008, 4096 00020000:00200000:1.0:1789760771.662294:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.662313:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16947, 16947] 00000008:00010000:1.0:1789760771.662584:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.662591:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0430, osc lock ffff93f852124980, flags 0x0 00000080:00200000:1.0:1789760771.662604:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69414912, 69419008) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.662618:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69414912=0x4233000, pages 1 (max 131072), lockless, parallel 00000100:00100000:3.0:1789760771.696428:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984256 gen 3 00000008:00010000:3.0:1789760771.696482:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760771.696504:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.696516:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.696536:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760771.696574:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565399040:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.697633:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565399168:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.707540:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984257 gen 3 00000008:00010000:3.0:1789760771.707602:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760771.707611:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.707622:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.707697:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760771.707727:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565399168:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.709027:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565399296:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.717963:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984258 gen 3 00000008:00010000:3.0:1789760771.718013:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760771.718022:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.718033:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.718056:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760771.718087:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565399296:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.719992:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760771.720018:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565399424:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.733857:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984259 gen 3 00000080:00200000:3.0:1789760771.735298:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16951, 16951] 00000080:00200000:3.0:1789760771.735317:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760771.735323:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760771.735339:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69435392 i_blocks 135640 00000020:00200000:3.0:1789760771.735342:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760771.735352:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760771.735418:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69435392, count: 4096 00000080:00200000:3.0:1789760771.735426:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69435392, bytes: 4096 00000080:00200000:3.0:1789760771.735441:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760771.735450:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760771.735452:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69435392, 69439488) 0 0 00000080:00200000:3.0:1789760771.735456:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760771.735474:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16952, 16952] 00020000:00200000:3.0:1789760771.736016:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69435392 chunk: [69435392, 69439488) 69439488, 4096 00020000:00200000:3.0:1789760771.736025:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:1.0:1789760771.737098:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760771.737191:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565399552:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760771.746043:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16952, 16952] 00000080:00200000:1.0:1789760771.746060:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.746066:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.746082:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69439488 i_blocks 135648 00000020:00200000:1.0:1789760771.746087:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.746096:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.746143:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69439488, count: 4096 00000080:00200000:1.0:1789760771.746151:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69439488, bytes: 4096 00000080:00200000:1.0:1789760771.746162:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.746171:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.746173:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69439488, 69443584) 0 0 00000080:00200000:1.0:1789760771.746178:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.746196:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16953, 16953] 00020000:00200000:1.0:1789760771.746711:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69439488 chunk: [69439488, 69443584) 69443584, 4096 00020000:00200000:1.0:1789760771.746718:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.746741:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16953, 16953] 00000008:00010000:1.0:1789760771.746760:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.746766:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760771.746779:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69439488, 69443584) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.746794:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69439488=0x4239000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.746857:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.746861:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69443584 00000020:00010000:1.0:1789760771.746869:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.746873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.746876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.746879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.746883:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.746891:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.746898:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.746930:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760771.747056:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 69439488<>69443584 00000080:00200000:1.0:1789760771.759236:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16953, 16953] 00000080:00200000:1.0:1789760771.759252:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.759258:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.759274:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69443584 i_blocks 135656 00000020:00200000:1.0:1789760771.759278:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.759288:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.759322:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69443584, count: 4096 00000080:00200000:1.0:1789760771.759329:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69443584, bytes: 4096 00000080:00200000:1.0:1789760771.759339:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.759347:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.759350:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69443584, 69447680) 0 0 00000080:00200000:1.0:1789760771.759354:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.759372:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16954, 16954] 00020000:00200000:1.0:1789760771.759381:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69443584 chunk: [69443584, 69447680) 69447680, 4096 00020000:00200000:1.0:1789760771.759385:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.759404:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16954, 16954] 00000008:00010000:1.0:1789760771.759422:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.759428:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760771.759441:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69443584, 69447680) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.759455:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69443584=0x423a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.759793:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.759797:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69447680 00000020:00010000:1.0:1789760771.759805:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.759809:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.759812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.759816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.759819:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.759826:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.759833:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.759852:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760771.759987:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 69443584<>69447680 00000080:00200000:1.0:1789760771.769885:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16954, 16954] 00000080:00200000:1.0:1789760771.769905:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.769913:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.769930:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69447680 i_blocks 135664 00000020:00200000:1.0:1789760771.769934:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.769945:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.769999:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69447680, count: 4096 00000080:00200000:1.0:1789760771.770008:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69447680, bytes: 4096 00000080:00200000:1.0:1789760771.770019:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.770028:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.770031:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69447680, 69451776) 0 0 00000080:00200000:1.0:1789760771.770036:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.770057:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16955, 16955] 00020000:00200000:1.0:1789760771.770066:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69447680 chunk: [69447680, 69451776) 69451776, 4096 00020000:00200000:1.0:1789760771.770071:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.770089:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16955, 16955] 00000008:00010000:1.0:1789760771.770108:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.770114:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760771.770124:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69447680, 69451776) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.770138:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69447680=0x423b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.770196:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.770335:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69451776 00000020:00010000:1.0:1789760771.770345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.770349:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.770352:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.770355:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.770358:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.770365:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.770371:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.770391:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760771.770642:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 69447680<>69451776 00000080:00200000:1.0:1789760771.780892:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16955, 16955] 00000080:00200000:1.0:1789760771.780917:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.780925:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.780942:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69451776 i_blocks 135672 00000020:00200000:1.0:1789760771.781485:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.781504:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.781544:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69451776, count: 4096 00000080:00200000:1.0:1789760771.781553:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69451776, bytes: 4096 00000080:00200000:1.0:1789760771.781593:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.781603:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.781606:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69451776, 69455872) 0 0 00000080:00200000:1.0:1789760771.781611:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.781630:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16956, 16956] 00020000:00200000:1.0:1789760771.781639:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69451776 chunk: [69451776, 69455872) 69455872, 4096 00020000:00200000:1.0:1789760771.781693:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.781718:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16956, 16956] 00000008:00010000:1.0:1789760771.781737:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.781744:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760771.781757:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69451776, 69455872) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.781772:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69451776=0x423c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.781845:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.781848:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69455872 00000020:00010000:1.0:1789760771.781857:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.781860:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.781863:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.781867:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.781870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.781879:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.781885:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.781913:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760771.782055:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 69451776<>69455872 00000100:00100000:1.0:1789760771.782114:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760771.782140:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565400064:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760771.795212:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16956, 16956] 00000080:00200000:1.0:1789760771.795237:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.795243:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.795259:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69455872 i_blocks 135680 00000020:00200000:1.0:1789760771.795263:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.795273:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.795307:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69455872, count: 4096 00000080:00200000:1.0:1789760771.795314:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69455872, bytes: 4096 00000080:00200000:1.0:1789760771.795324:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.795333:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.795335:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69455872, 69459968) 0 0 00000080:00200000:1.0:1789760771.795339:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.795357:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16957, 16957] 00020000:00200000:1.0:1789760771.795366:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69455872 chunk: [69455872, 69459968) 69459968, 4096 00020000:00200000:1.0:1789760771.795370:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.795391:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16957, 16957] 00000008:00010000:1.0:1789760771.795408:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.795415:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760771.795426:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69455872, 69459968) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.795440:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69455872=0x423d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.795501:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.795504:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69459968 00000020:00010000:1.0:1789760771.795512:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.795516:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.795519:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.795522:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.795525:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.795533:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.795539:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.795574:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760771.795746:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 69455872<>69459968 00000080:00200000:1.0:1789760771.806685:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16957, 16957] 00000080:00200000:1.0:1789760771.806702:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.806708:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.806724:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69459968 i_blocks 135688 00000020:00200000:1.0:1789760771.806830:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.806841:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.806877:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69459968, count: 4096 00000080:00200000:1.0:1789760771.806885:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69459968, bytes: 4096 00000080:00200000:1.0:1789760771.806895:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.806905:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.806907:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69459968, 69464064) 0 0 00000080:00200000:1.0:1789760771.806912:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.806930:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16958, 16958] 00020000:00200000:1.0:1789760771.806939:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69459968 chunk: [69459968, 69464064) 69464064, 4096 00020000:00200000:1.0:1789760771.806944:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.806963:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16958, 16958] 00000008:00010000:1.0:1789760771.806982:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.806988:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760771.806999:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69459968, 69464064) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.807013:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69459968=0x423e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.807071:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.807074:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69464064 00000020:00010000:1.0:1789760771.807083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.807086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.807090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.807093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.807096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.807104:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.807111:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.807130:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760771.807260:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 69459968<>69464064 00000100:00100000:1.0:1789760771.820619:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984266 gen 3 00000008:00010000:1.0:1789760771.820686:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760771.820695:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760771.820706:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760771.820732:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760771.820762:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565400320:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.821609:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760771.821627:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565400448:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.830710:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984267 gen 3 00000008:00010000:3.0:1789760771.830765:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760771.830775:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.830787:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.831929:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760771.831975:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565400448:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760771.835603:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:3.0:1789760771.835774:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 69468160<>69472256 00000080:00200000:1.0:1789760771.857229:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16960, 16960] 00000080:00200000:1.0:1789760771.857245:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.857251:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.857267:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69472256 i_blocks 135712 00000020:00200000:1.0:1789760771.857271:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.857282:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.857317:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69472256, count: 4096 00000080:00200000:1.0:1789760771.857324:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69472256, bytes: 4096 00000080:00200000:1.0:1789760771.857334:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.857343:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.857345:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69472256, 69476352) 0 0 00000080:00200000:1.0:1789760771.857350:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.857366:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16961, 16961] 00020000:00200000:1.0:1789760771.857374:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69472256 chunk: [69472256, 69476352) 69476352, 4096 00020000:00200000:1.0:1789760771.857380:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.857399:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16961, 16961] 00000008:00010000:1.0:1789760771.857415:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.857421:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760771.857571:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69472256, 69476352) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.857586:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69472256=0x4241000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.857747:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.857750:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69476352 00000020:00010000:1.0:1789760771.857760:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.857763:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.857767:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.857771:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.857774:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.857783:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.857790:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.857809:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760771.857935:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 69472256<>69476352 00000100:00100000:1.0:1789760771.870961:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984269 gen 3 00000008:00010000:1.0:1789760771.871014:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760771.871023:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760771.871035:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760771.871061:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760771.871092:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565400704:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.871816:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565400832:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760771.885141:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16962, 16962] 00000080:00200000:1.0:1789760771.885157:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.885163:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.885180:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69480448 i_blocks 135728 00000020:00200000:1.0:1789760771.885184:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.885194:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.885229:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69480448, count: 4096 00000080:00200000:1.0:1789760771.885236:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69480448, bytes: 4096 00000080:00200000:1.0:1789760771.885249:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.885257:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.885260:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69480448, 69484544) 0 0 00000080:00200000:1.0:1789760771.885266:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.885283:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16963, 16963] 00020000:00200000:1.0:1789760771.885291:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69480448 chunk: [69480448, 69484544) 69484544, 4096 00020000:00200000:1.0:1789760771.885296:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.885315:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16963, 16963] 00000008:00010000:1.0:1789760771.885332:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.885337:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760771.885348:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69480448, 69484544) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.885362:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69480448=0x4243000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.885417:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.885421:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69484544 00000020:00010000:1.0:1789760771.885556:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.885570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.885574:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.885577:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.885580:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.885588:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.885766:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.885789:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760771.885993:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 69480448<>69484544 00000100:00100000:1.0:1789760771.895073:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984271 gen 3 00000008:00010000:1.0:1789760771.895124:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760771.895132:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760771.895142:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760771.895223:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760771.895254:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565400960:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760771.906885:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16964, 16964] 00000080:00200000:1.0:1789760771.906901:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.906975:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.906992:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69488640 i_blocks 135744 00000020:00200000:1.0:1789760771.906996:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.907007:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.907044:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69488640, count: 4096 00000080:00200000:1.0:1789760771.907052:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69488640, bytes: 4096 00000080:00200000:1.0:1789760771.907063:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.907071:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.907073:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69488640, 69492736) 0 0 00000080:00200000:1.0:1789760771.907077:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.907095:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16965, 16965] 00020000:00200000:1.0:1789760771.907377:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69488640 chunk: [69488640, 69492736) 69492736, 4096 00020000:00200000:1.0:1789760771.907385:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760771.907408:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16965, 16965] 00000008:00010000:1.0:1789760771.907428:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.907434:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760771.907446:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69488640, 69492736) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.907460:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69488640=0x4245000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.907528:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.907531:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69492736 00000020:00010000:1.0:1789760771.907539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.907543:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.907547:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.907551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.907554:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.907572:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.907580:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.907798:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760771.908004:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 69488640<>69492736 00000100:00100000:1.0:1789760771.908059:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760771.908080:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565401216:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760771.918975:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984273 gen 3 00000008:00010000:1.0:1789760771.919027:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760771.919035:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760771.919045:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760771.919075:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760771.919105:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565401216:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.923705:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565401344:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.940541:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984274 gen 3 00000008:00010000:3.0:1789760771.940613:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760771.940622:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.940634:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.940725:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16966, 16966] 00000080:00200000:1.0:1789760771.940745:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760771.940751:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760771.940766:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69496832 i_blocks 135760 00000020:00200000:1.0:1789760771.940771:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760771.940781:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760771.940820:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69496832, count: 4096 00000080:00200000:1.0:1789760771.940828:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69496832, bytes: 4096 00000080:00200000:1.0:1789760771.940841:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760771.940852:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760771.940854:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69496832, 69500928) 0 0 00000080:00200000:1.0:1789760771.940859:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760771.940888:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16967, 16967] 00020000:00200000:1.0:1789760771.940898:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69496832 chunk: [69496832, 69500928) 69500928, 4096 00020000:00200000:1.0:1789760771.940903:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:3.0:1789760771.940920:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000080:00200000:1.0:1789760771.940926:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16967, 16967] 00000008:00010000:1.0:1789760771.940946:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760771.940954:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000100:00100000:3.0:1789760771.940954:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565401344:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760771.940968:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69496832, 69500928) append 0 append lockpos 0 00000080:00200000:1.0:1789760771.940983:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69496832=0x4247000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760771.941044:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760771.941047:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69500928 00000020:00010000:1.0:1789760771.941055:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760771.941059:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.941062:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760771.941067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760771.941070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760771.941078:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760771.941084:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760771.941101:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760771.941219:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 69496832<>69500928 00000100:00100000:1.0:1789760771.941261:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760771.941278:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565401472:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760771.954063:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984275 gen 3 00000008:00010000:1.0:1789760771.954110:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760771.954118:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760771.954129:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760771.954154:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760771.954184:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565401472:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760771.972058:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984276 gen 3 00000008:00010000:1.0:1789760771.972132:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760771.972140:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760771.972152:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760771.972179:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:1.0:1789760771.972209:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565401600:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.975217:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565401728:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.988106:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984277 gen 3 00000008:00010000:3.0:1789760771.988156:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760771.988164:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760771.988176:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760771.988257:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760771.988287:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565401728:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760771.989016:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760771.989033:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565401856:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.001124:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984278 gen 3 00000008:00010000:3.0:1789760772.001182:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760772.001191:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.001201:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.001225:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760772.001259:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565401856:192.168.203.147@tcp:4:dd.0 00000020:00200000:1.0:1789760772.002442:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.002446:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69517312 00000020:00010000:1.0:1789760772.002456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.002459:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.002463:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d4b0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.002467:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.002470:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d4b0 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.002478:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.002487:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.002504:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760772.002816:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 69513216<>69517312 00000080:00200000:3.0:1789760772.039539:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16972, 16972] 00000080:00200000:3.0:1789760772.039567:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760772.039573:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760772.039589:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69521408 i_blocks 135808 00000020:00200000:3.0:1789760772.039594:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760772.039605:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760772.039685:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69521408, count: 4096 00000080:00200000:3.0:1789760772.039694:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69521408, bytes: 4096 00000080:00200000:3.0:1789760772.039707:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760772.039717:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760772.039719:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69521408, 69525504) 0 0 00000080:00200000:3.0:1789760772.039724:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760772.039742:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16973, 16973] 00020000:00200000:3.0:1789760772.039751:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69521408 chunk: [69521408, 69525504) 69525504, 4096 00020000:00200000:3.0:1789760772.039756:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760772.039778:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16973, 16973] 00000008:00010000:3.0:1789760772.039798:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760772.039804:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760772.039816:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69521408, 69525504) append 0 append lockpos 0 00000080:00200000:3.0:1789760772.039830:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69521408=0x424d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760772.039895:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760772.039898:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69525504 00000020:00010000:3.0:1789760772.039906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760772.039909:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.039913:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.039917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760772.039920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760772.039928:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760772.039949:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760772.039967:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760772.040096:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 69521408<>69525504 00000100:00100000:3.0:1789760772.040141:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760772.040159:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565402240:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.055358:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984281 gen 3 00000008:00010000:3.0:1789760772.055411:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:3.0:1789760772.055419:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.055430:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.055499:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:3.0:1789760772.055530:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565402240:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.056295:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565402368:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760772.068731:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16974, 16974] 00000080:00200000:3.0:1789760772.068755:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760772.068761:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760772.068777:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69529600 i_blocks 135824 00000020:00200000:3.0:1789760772.068782:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760772.068798:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760772.068830:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69529600, count: 4096 00000080:00200000:3.0:1789760772.068838:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69529600, bytes: 4096 00000080:00200000:3.0:1789760772.068851:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760772.068861:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760772.068863:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69529600, 69533696) 0 0 00000080:00200000:3.0:1789760772.068869:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760772.068885:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16975, 16975] 00020000:00200000:3.0:1789760772.068894:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69529600 chunk: [69529600, 69533696) 69533696, 4096 00020000:00200000:3.0:1789760772.068899:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760772.068931:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16975, 16975] 00000008:00010000:3.0:1789760772.068950:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760772.068956:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760772.068968:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69529600, 69533696) append 0 append lockpos 0 00000080:00200000:3.0:1789760772.068982:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69529600=0x424f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760772.069044:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760772.069046:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69533696 00000020:00010000:3.0:1789760772.069054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760772.069058:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.069060:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.069063:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760772.069066:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760772.069075:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760772.069082:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760772.069102:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760772.069225:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 69529600<>69533696 00000080:00200000:3.0:1789760772.081447:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16975, 16975] 00000080:00200000:3.0:1789760772.081463:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760772.081469:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760772.082253:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565402624:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.093097:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565402752:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.108831:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565402880:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.129147:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984286 gen 3 00000008:00010000:3.0:1789760772.129204:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760772.129213:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.129226:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.129253:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760772.129281:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565402880:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760772.147999:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16979, 16979] 00000080:00200000:3.0:1789760772.148015:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760772.148020:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760772.148035:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69550080 i_blocks 135864 00000020:00200000:3.0:1789760772.148039:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760772.148051:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760772.148111:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69550080, count: 4096 00000080:00200000:3.0:1789760772.148118:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69550080, bytes: 4096 00000080:00200000:3.0:1789760772.148131:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760772.148139:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760772.148142:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69550080, 69554176) 0 0 00000080:00200000:3.0:1789760772.148146:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760772.148162:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16980, 16980] 00020000:00200000:3.0:1789760772.148170:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69550080 chunk: [69550080, 69554176) 69554176, 4096 00020000:00200000:3.0:1789760772.148175:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760772.148201:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16980, 16980] 00000008:00010000:3.0:1789760772.148220:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760772.148226:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760772.148238:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69550080, 69554176) append 0 append lockpos 0 00000080:00200000:3.0:1789760772.148253:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69550080=0x4254000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760772.148309:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760772.148312:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69554176 00000020:00010000:3.0:1789760772.148320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760772.148323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.148326:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.148329:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760772.148333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760772.148340:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760772.148348:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760772.148365:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760772.149048:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 69550080<>69554176 00000100:00100000:3.0:1789760772.149101:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760772.149118:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565403136:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.170588:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984288 gen 3 00000008:00010000:3.0:1789760772.170638:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:3.0:1789760772.170944:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.170956:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.171021:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:3.0:1789760772.171049:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565403136:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.175568:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760772.175610:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565403264:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760772.189903:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16981, 16981] 00000080:00200000:3.0:1789760772.189920:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760772.189926:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760772.189941:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69558272 i_blocks 135880 00000020:00200000:3.0:1789760772.189946:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760772.189957:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760772.189991:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69558272, count: 4096 00000080:00200000:3.0:1789760772.189999:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69558272, bytes: 4096 00000080:00200000:3.0:1789760772.190009:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760772.190019:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760772.190021:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69558272, 69562368) 0 0 00000080:00200000:3.0:1789760772.190026:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760772.190044:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16982, 16982] 00020000:00200000:3.0:1789760772.190053:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69558272 chunk: [69558272, 69562368) 69562368, 4096 00020000:00200000:3.0:1789760772.190058:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760772.190077:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16982, 16982] 00000008:00010000:3.0:1789760772.190095:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760772.190103:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760772.190115:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69558272, 69562368) append 0 append lockpos 0 00000080:00200000:3.0:1789760772.190128:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69558272=0x4256000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760772.190175:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760772.190178:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69562368 00000020:00010000:3.0:1789760772.190186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760772.190191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.190194:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.190198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760772.190200:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760772.190209:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760772.190216:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760772.207364:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16982, 16982] 00000080:00200000:3.0:1789760772.207382:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760772.207388:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760772.207404:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69562368 i_blocks 135888 00000020:00200000:3.0:1789760772.207408:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760772.207419:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760772.207468:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69562368, count: 4096 00000080:00200000:3.0:1789760772.207477:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69562368, bytes: 4096 00000080:00200000:3.0:1789760772.207502:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760772.207511:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760772.207514:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69562368, 69566464) 0 0 00000080:00200000:3.0:1789760772.207519:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760772.207536:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16983, 16983] 00020000:00200000:3.0:1789760772.207546:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69562368 chunk: [69562368, 69566464) 69566464, 4096 00020000:00200000:3.0:1789760772.207567:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760772.207589:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16983, 16983] 00000008:00010000:3.0:1789760772.207608:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760772.207613:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760772.207625:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69562368, 69566464) append 0 append lockpos 0 00000080:00200000:3.0:1789760772.207639:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69562368=0x4257000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760772.207741:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760772.207744:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69566464 00000020:00010000:3.0:1789760772.207752:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760772.207756:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.207759:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.207763:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760772.207767:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760772.207775:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760772.207782:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760772.207800:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760772.208075:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 69562368<>69566464 00000100:00100000:3.0:1789760772.208128:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565403520:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.220014:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565403648:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760772.235232:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16984, 16984] 00000080:00200000:1.0:1789760772.235249:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.235254:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760772.235271:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69570560 i_blocks 135904 00000020:00200000:1.0:1789760772.235275:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760772.235286:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760772.235319:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69570560, count: 4096 00000080:00200000:1.0:1789760772.235327:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69570560, bytes: 4096 00000080:00200000:1.0:1789760772.235338:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760772.235347:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760772.235349:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69570560, 69574656) 0 0 00000080:00200000:1.0:1789760772.235353:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760772.235371:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16985, 16985] 00020000:00200000:1.0:1789760772.235379:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69570560 chunk: [69570560, 69574656) 69574656, 4096 00020000:00200000:1.0:1789760772.235384:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760772.235403:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16985, 16985] 00000008:00010000:1.0:1789760772.235422:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760772.235427:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760772.235440:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69570560, 69574656) append 0 append lockpos 0 00000080:00200000:1.0:1789760772.235453:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69570560=0x4259000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760772.235522:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.235525:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69574656 00000020:00010000:1.0:1789760772.235533:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.235698:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.235702:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.235706:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.235709:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.235719:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.235727:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.235747:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760772.235874:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 69570560<>69574656 00000080:00200000:1.0:1789760772.259836:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16985, 16985] 00000080:00200000:1.0:1789760772.259855:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.259861:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760772.259878:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69574656 i_blocks 135912 00000020:00200000:1.0:1789760772.259883:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760772.259894:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760772.259927:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69574656, count: 4096 00000080:00200000:1.0:1789760772.259934:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69574656, bytes: 4096 00000080:00200000:1.0:1789760772.259946:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760772.259955:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760772.259957:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69574656, 69578752) 0 0 00000080:00200000:1.0:1789760772.259961:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760772.259979:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16986, 16986] 00020000:00200000:1.0:1789760772.259988:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69574656 chunk: [69574656, 69578752) 69578752, 4096 00020000:00200000:1.0:1789760772.259993:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760772.260014:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16986, 16986] 00000008:00010000:1.0:1789760772.260032:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760772.260037:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760772.260050:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69574656, 69578752) append 0 append lockpos 0 00000080:00200000:1.0:1789760772.260062:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69574656=0x425a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760772.260124:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.260127:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69578752 00000020:00010000:1.0:1789760772.260135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.260139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.260143:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.260147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.260149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.260157:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.260165:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.260183:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760772.260321:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 69574656<>69578752 00000080:00200000:1.0:1789760772.269199:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16986, 16986] 00000080:00200000:1.0:1789760772.269216:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.269222:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760772.269239:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69578752 i_blocks 135920 00000020:00200000:1.0:1789760772.269242:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760772.269252:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760772.269286:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69578752, count: 4096 00000080:00200000:1.0:1789760772.269293:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69578752, bytes: 4096 00000080:00200000:1.0:1789760772.269303:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760772.269312:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760772.269315:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69578752, 69582848) 0 0 00000080:00200000:1.0:1789760772.269320:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760772.269337:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16987, 16987] 00020000:00200000:1.0:1789760772.269346:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69578752 chunk: [69578752, 69582848) 69582848, 4096 00020000:00200000:1.0:1789760772.269351:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760772.269372:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16987, 16987] 00000008:00010000:1.0:1789760772.269391:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760772.269397:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760772.269410:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69578752, 69582848) append 0 append lockpos 0 00000080:00200000:1.0:1789760772.269425:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69578752=0x425b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760772.269485:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.269622:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69582848 00000020:00010000:1.0:1789760772.269633:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.269637:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.269640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.269672:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.269675:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.269684:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.269692:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.269712:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760772.269841:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 69578752<>69582848 00000080:00200000:1.0:1789760772.287335:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16987, 16987] 00000080:00200000:1.0:1789760772.287364:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.287372:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760772.287390:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69582848 i_blocks 135928 00000020:00200000:1.0:1789760772.287396:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760772.287412:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760772.287448:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69582848, count: 4096 00000080:00200000:1.0:1789760772.287457:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69582848, bytes: 4096 00000080:00200000:1.0:1789760772.287472:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760772.287481:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760772.287484:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69582848, 69586944) 0 0 00000080:00200000:1.0:1789760772.287488:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760772.287518:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16988, 16988] 00020000:00200000:1.0:1789760772.287527:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69582848 chunk: [69582848, 69586944) 69586944, 4096 00020000:00200000:1.0:1789760772.287532:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760772.287683:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16988, 16988] 00000008:00010000:1.0:1789760772.287703:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760772.287709:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760772.287721:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69582848, 69586944) append 0 append lockpos 0 00000080:00200000:1.0:1789760772.287737:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69582848=0x425c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760772.287813:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.287817:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69586944 00000020:00010000:1.0:1789760772.287825:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.287829:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.287832:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.287836:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.287839:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.287847:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.287856:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.287887:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760772.288017:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 69582848<>69586944 00000080:00200000:1.0:1789760772.306232:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16988, 16988] 00000080:00200000:1.0:1789760772.306248:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.306254:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760772.306269:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69586944 i_blocks 135936 00000020:00200000:1.0:1789760772.306273:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760772.306283:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760772.306321:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69586944, count: 4096 00000080:00200000:1.0:1789760772.306328:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69586944, bytes: 4096 00000080:00200000:1.0:1789760772.306343:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760772.306352:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760772.306355:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69586944, 69591040) 0 0 00000080:00200000:1.0:1789760772.306359:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760772.306375:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16989, 16989] 00020000:00200000:1.0:1789760772.306383:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69586944 chunk: [69586944, 69591040) 69591040, 4096 00020000:00200000:1.0:1789760772.306388:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760772.306408:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16989, 16989] 00000008:00010000:1.0:1789760772.306427:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760772.306433:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760772.306445:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69586944, 69591040) append 0 append lockpos 0 00000080:00200000:1.0:1789760772.306459:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69586944=0x425d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760772.306516:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.306518:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69591040 00000020:00010000:1.0:1789760772.306526:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.306530:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.306533:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.306537:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.306540:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.306548:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.307044:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.307068:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760772.307205:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 69586944<>69591040 00000080:00200000:1.0:1789760772.321208:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16989, 16989] 00000080:00200000:1.0:1789760772.321226:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.321232:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760772.321248:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69591040 i_blocks 135944 00000020:00200000:1.0:1789760772.321252:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760772.321262:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760772.321299:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69591040, count: 4096 00000080:00200000:1.0:1789760772.321307:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69591040, bytes: 4096 00000080:00200000:1.0:1789760772.321318:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760772.321327:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760772.321330:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69591040, 69595136) 0 0 00000080:00200000:1.0:1789760772.321333:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760772.321350:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16990, 16990] 00020000:00200000:1.0:1789760772.321359:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69591040 chunk: [69591040, 69595136) 69595136, 4096 00020000:00200000:1.0:1789760772.321364:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760772.321384:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16990, 16990] 00000008:00010000:1.0:1789760772.321402:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760772.321408:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760772.321419:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69591040, 69595136) append 0 append lockpos 0 00000080:00200000:1.0:1789760772.321433:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69591040=0x425e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760772.322141:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.322146:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69595136 00000020:00010000:1.0:1789760772.322156:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.322160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.322164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.322168:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.322171:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.322180:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.322188:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.322212:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760772.322343:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 69591040<>69595136 00000080:00200000:1.0:1789760772.334255:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16990, 16990] 00000080:00200000:1.0:1789760772.334273:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.334279:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:1.0:1789760772.350001:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760772.353179:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565404672:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760772.372103:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16992, 16992] 00000080:00200000:1.0:1789760772.372119:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.372126:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760772.372141:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69603328 i_blocks 135968 00000020:00200000:1.0:1789760772.372145:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760772.372156:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760772.372191:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69603328, count: 4096 00000080:00200000:1.0:1789760772.372199:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69603328, bytes: 4096 00000080:00200000:1.0:1789760772.372210:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760772.372219:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760772.372222:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69603328, 69607424) 0 0 00000080:00200000:1.0:1789760772.372226:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760772.372244:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16993, 16993] 00020000:00200000:1.0:1789760772.372254:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69603328 chunk: [69603328, 69607424) 69607424, 4096 00020000:00200000:1.0:1789760772.372258:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760772.372277:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16993, 16993] 00000008:00010000:1.0:1789760772.372295:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760772.372301:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760772.372312:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69603328, 69607424) append 0 append lockpos 0 00000080:00200000:1.0:1789760772.372325:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69603328=0x4261000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760772.372384:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.372387:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69607424 00000020:00010000:1.0:1789760772.372395:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.372399:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.372403:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.372407:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.372409:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.372416:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.372423:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.372439:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760772.374630:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 69603328<>69607424 00000080:00200000:1.0:1789760772.391934:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [16993, 16993] 00000080:00200000:1.0:1789760772.391949:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.391956:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760772.391971:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69607424 i_blocks 135976 00000020:00200000:1.0:1789760772.391975:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760772.391985:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760772.392019:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69607424, count: 4096 00000080:00200000:1.0:1789760772.392027:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69607424, bytes: 4096 00000080:00200000:1.0:1789760772.392037:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760772.392047:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760772.392049:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69607424, 69611520) 0 0 00000080:00200000:1.0:1789760772.392053:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760772.392070:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [16994, 16994] 00020000:00200000:1.0:1789760772.392079:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69607424 chunk: [69607424, 69611520) 69611520, 4096 00020000:00200000:1.0:1789760772.392085:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760772.392105:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [16994, 16994] 00000008:00010000:1.0:1789760772.392123:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760772.393278:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760772.393301:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69607424, 69611520) append 0 append lockpos 0 00000080:00200000:1.0:1789760772.393456:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69607424=0x4262000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760772.393547:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.393550:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69611520 00000020:00010000:1.0:1789760772.393568:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.393573:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.393576:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.393580:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.393582:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.393592:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.393600:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.393808:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760772.393943:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 69607424<>69611520 00000100:00100000:1.0:1789760772.409284:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984302 gen 3 00000008:00010000:1.0:1789760772.409347:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:1.0:1789760772.409356:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.409368:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.409392:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:1.0:1789760772.409422:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565404928:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.410702:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565405056:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.424848:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984303 gen 3 00000008:00010000:3.0:1789760772.424968:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760772.424977:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.424989:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.425085:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760772.425193:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565405056:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.426278:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760772.426299:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565405184:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.439192:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984304 gen 3 00000008:00010000:3.0:1789760772.439246:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760772.439256:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.439268:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.439295:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760772.439328:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565405184:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.440314:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760772.440336:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565405312:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.448308:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984305 gen 3 00000008:00010000:3.0:1789760772.448360:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441890c0 00000020:00200000:3.0:1789760772.448369:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.448381:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.448412:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441890c0 00000100:00100000:3.0:1789760772.448444:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565405312:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.460810:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984306 gen 3 00000008:00010000:3.0:1789760772.460859:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760772.460868:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.460890:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.460914:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760772.460946:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565405440:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.461491:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565405568:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.473478:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984307 gen 3 00000008:00010000:3.0:1789760772.473563:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760772.473572:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.473583:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.473605:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760772.473636:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565405568:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.477406:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760772.477445:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565405696:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.486945:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984308 gen 3 00000008:00010000:3.0:1789760772.486997:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760772.487006:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.487018:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.487044:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760772.487076:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565405696:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.487947:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565405824:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.495984:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984309 gen 3 00000008:00010000:3.0:1789760772.496038:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760772.496047:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.496057:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.496083:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760772.496114:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565405824:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.498430:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565405952:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.513862:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984310 gen 3 00000008:00010000:3.0:1789760772.513918:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441890c0 00000020:00200000:3.0:1789760772.513927:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.513952:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.513978:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441890c0 00000100:00100000:3.0:1789760772.514009:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565405952:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.514885:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565406080:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760772.535504:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17003, 17003] 00000080:00200000:1.0:1789760772.535530:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.535538:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760772.535566:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69648384 i_blocks 136056 00000020:00200000:1.0:1789760772.535570:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760772.535585:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760772.535623:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69648384, count: 4096 00000080:00200000:1.0:1789760772.535632:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69648384, bytes: 4096 00000080:00200000:1.0:1789760772.535664:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760772.535674:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760772.535676:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69648384, 69652480) 0 0 00000080:00200000:1.0:1789760772.535681:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760772.535700:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17004, 17004] 00020000:00200000:1.0:1789760772.535709:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69648384 chunk: [69648384, 69652480) 69652480, 4096 00020000:00200000:1.0:1789760772.535714:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760772.535735:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17004, 17004] 00000008:00010000:1.0:1789760772.535755:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760772.535761:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760772.535832:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69648384, 69652480) append 0 append lockpos 0 00000080:00200000:1.0:1789760772.535848:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69648384=0x426c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760772.535911:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.535913:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69652480 00000020:00010000:1.0:1789760772.535921:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.535925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.535928:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.535932:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.535935:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.535942:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.535948:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.535972:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760772.536206:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 69648384<>69652480 00000100:00100000:1.0:1789760772.550251:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984312 gen 3 00000008:00010000:1.0:1789760772.550302:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:1.0:1789760772.550320:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.550333:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.550424:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:1.0:1789760772.550456:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565406208:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.578179:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760772.578221:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565406464:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.594411:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984314 gen 3 00000008:00010000:3.0:1789760772.594464:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760772.594472:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.594484:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.594515:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760772.594545:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565406464:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.595439:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565406592:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.608916:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984315 gen 3 00000008:00010000:3.0:1789760772.608968:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760772.608977:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.608987:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.609015:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760772.609055:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565406592:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.610530:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565406720:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.621978:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984316 gen 3 00000008:00010000:3.0:1789760772.622028:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760772.622036:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.622048:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.622076:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760772.622104:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565406720:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.626695:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565406848:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760772.638863:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760772.640224:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984317 gen 3 00000008:00010000:3.0:1789760772.640275:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760772.640284:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.640295:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.640376:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760772.640410:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565406848:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.643684:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565406976:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.655829:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565407104:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.670572:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984319 gen 3 00000008:00010000:1.0:1789760772.670622:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760772.670630:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.670777:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.670798:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760772.670820:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565407104:192.168.203.147@tcp:4:dd.0 00000008:00010000:1.0:1789760772.671459:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:1.0:1789760772.671608:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 69681152<>69685248 00000100:00100000:1.0:1789760772.671798:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565407232:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.686615:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984320 gen 3 00000008:00010000:1.0:1789760772.686763:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760772.686773:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.686786:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.686808:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760772.686839:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565407232:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.688856:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565407360:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.698878:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984321 gen 3 00000008:00010000:1.0:1789760772.698929:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760772.698938:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.698949:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.698970:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760772.699000:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565407360:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.700112:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565407488:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.712270:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984322 gen 3 00000008:00010000:1.0:1789760772.712323:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760772.712332:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.712343:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.712365:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760772.712396:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565407488:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.715165:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565407616:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.726340:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984323 gen 3 00000008:00010000:1.0:1789760772.726392:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760772.726400:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.726411:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.726434:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:1.0:1789760772.726464:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565407616:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.727198:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565407744:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.740147:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984324 gen 3 00000008:00010000:1.0:1789760772.740198:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760772.740206:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.740218:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.740250:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760772.740282:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565407744:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.741095:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565407872:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.758584:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565408000:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.775366:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984326 gen 3 00000008:00010000:1.0:1789760772.775416:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760772.775425:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.775437:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.775455:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760772.775486:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565408000:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.777932:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760772.777970:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565408128:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.789079:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984327 gen 3 00000008:00010000:1.0:1789760772.789135:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760772.789144:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.789156:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.789181:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:1.0:1789760772.789213:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565408128:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.793449:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565408256:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.809055:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984328 gen 3 00000008:00010000:1.0:1789760772.809107:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760772.809115:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.809126:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.809152:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760772.809184:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565408256:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.811429:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760772.811454:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565408384:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.831749:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984329 gen 3 00000008:00010000:1.0:1789760772.831798:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760772.831806:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.831816:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.831839:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760772.831868:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565408384:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.837487:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565408512:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.848166:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984330 gen 3 00000008:00010000:1.0:1789760772.848215:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760772.848223:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.848235:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.848321:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760772.848353:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565408512:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.852564:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565408640:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.865130:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984331 gen 3 00000008:00010000:3.0:1789760772.865171:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760772.865179:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.865188:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.865217:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760772.865246:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565408640:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.866243:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565408768:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.885469:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984332 gen 3 00000008:00010000:3.0:1789760772.885535:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760772.885544:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.885568:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.885603:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760772.885681:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565408768:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.892333:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565408896:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.906703:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984333 gen 3 00000008:00010000:3.0:1789760772.906755:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760772.906764:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.906775:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.906801:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760772.906833:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565408896:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.907919:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565409024:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.916564:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984334 gen 3 00000008:00010000:3.0:1789760772.916616:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760772.916624:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.916635:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.916745:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760772.916776:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565409024:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.919142:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565409152:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760772.929870:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984335 gen 3 00000008:00010000:3.0:1789760772.929986:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441890c0 00000020:00200000:3.0:1789760772.929996:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760772.930006:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760772.930031:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441890c0 00000100:00100000:3.0:1789760772.930060:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565409152:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.940518:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565409408:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.952190:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984337 gen 3 00000008:00010000:1.0:1789760772.952251:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760772.952260:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.952273:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.952298:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760772.952330:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565409408:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760772.966326:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984338 gen 3 00000008:00010000:1.0:1789760772.966377:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760772.966385:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.966396:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.966422:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:1.0:1789760772.966455:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565409536:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760772.981903:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17031, 17031] 00000080:00200000:1.0:1789760772.981920:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760772.981926:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760772.981942:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69763072 i_blocks 136280 00000020:00200000:1.0:1789760772.981945:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760772.981955:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760772.981991:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69763072, count: 4096 00000080:00200000:1.0:1789760772.981998:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69763072, bytes: 4096 00000080:00200000:1.0:1789760772.982009:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760772.982018:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760772.982020:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69763072, 69767168) 0 0 00000080:00200000:1.0:1789760772.982024:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760772.982041:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17032, 17032] 00020000:00200000:1.0:1789760772.982049:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69763072 chunk: [69763072, 69767168) 69767168, 4096 00020000:00200000:1.0:1789760772.982054:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760772.982073:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17032, 17032] 00000008:00010000:1.0:1789760772.982092:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760772.982098:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760772.982108:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69763072, 69767168) append 0 append lockpos 0 00000080:00200000:1.0:1789760772.982121:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69763072=0x4288000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760772.982180:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760772.982184:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69767168 00000020:00010000:1.0:1789760772.982192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760772.982195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.982199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760772.982202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760772.982205:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760772.982211:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760772.982218:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760772.982235:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760772.982358:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 69763072<>69767168 00000100:00100000:1.0:1789760772.996514:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984340 gen 3 00000008:00010000:1.0:1789760772.996574:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:1.0:1789760772.996582:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760772.996594:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760772.996618:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:1.0:1789760772.996694:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565409792:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760772.997163:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17032, 17032] 00000080:00200000:3.0:1789760772.997180:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760772.997186:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760772.997201:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69767168 i_blocks 136288 00000020:00200000:3.0:1789760772.997205:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760772.997215:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760772.997249:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69767168, count: 4096 00000080:00200000:3.0:1789760772.997257:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69767168, bytes: 4096 00000080:00200000:3.0:1789760772.997422:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760772.997432:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760772.997435:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69767168, 69771264) 0 0 00000080:00200000:3.0:1789760772.997440:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760772.997458:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17033, 17033] 00020000:00200000:3.0:1789760772.997468:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69767168 chunk: [69767168, 69771264) 69771264, 4096 00020000:00200000:3.0:1789760772.997473:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760772.997493:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17033, 17033] 00000008:00010000:3.0:1789760772.997511:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760772.997517:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760772.997529:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69767168, 69771264) append 0 append lockpos 0 00000080:00200000:3.0:1789760772.997542:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69767168=0x4289000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760772.997670:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760772.997674:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69771264 00000020:00010000:3.0:1789760772.997685:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760772.997688:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.997694:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760772.997698:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760772.997701:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760772.997709:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760772.997718:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760772.997739:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760772.997892:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 69767168<>69771264 00000100:00100000:3.0:1789760772.997946:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760772.997969:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565409920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.007559:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984341 gen 3 00000008:00010000:3.0:1789760773.007615:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760773.007623:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.007634:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.007700:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760773.007727:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565409920:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760773.025547:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17034, 17034] 00000080:00200000:3.0:1789760773.025572:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.025579:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.025596:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69775360 i_blocks 136304 00000020:00200000:3.0:1789760773.025601:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.025613:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.025693:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69775360, count: 4096 00000080:00200000:3.0:1789760773.025701:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69775360, bytes: 4096 00000080:00200000:3.0:1789760773.025713:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.025722:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.025725:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69775360, 69779456) 0 0 00000080:00200000:3.0:1789760773.025730:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.025749:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17035, 17035] 00020000:00200000:3.0:1789760773.025757:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69775360 chunk: [69775360, 69779456) 69779456, 4096 00020000:00200000:3.0:1789760773.025762:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.025784:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17035, 17035] 00000008:00010000:3.0:1789760773.025803:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.025809:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.025821:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69775360, 69779456) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.025836:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69775360=0x428b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.025896:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.025899:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69779456 00000020:00010000:3.0:1789760773.025907:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.025911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.025915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.025918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.025921:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.025929:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.025936:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.025962:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760773.026229:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 69775360<>69779456 00000100:00100000:3.0:1789760773.026574:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760773.026599:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565410176:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.040907:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984343 gen 3 00000008:00010000:3.0:1789760773.040970:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760773.040979:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.040992:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.041021:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760773.041058:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565410176:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760773.042532:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565410304:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760773.056053:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984344 gen 3 00000008:00010000:1.0:1789760773.056102:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760773.056110:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760773.056122:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760773.056185:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760773.056216:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565410304:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760773.057857:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565410432:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760773.072716:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984345 gen 3 00000008:00010000:1.0:1789760773.072766:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760773.072775:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760773.072786:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760773.072805:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:1.0:1789760773.072837:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565410432:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760773.074126:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760773.074205:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565410560:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760773.082804:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984346 gen 3 00000008:00010000:1.0:1789760773.082859:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760773.082869:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760773.082882:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760773.083062:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760773.083096:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565410560:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760773.084667:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17038, 17038] 00000080:00200000:3.0:1789760773.084696:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.084705:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.084722:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69791744 i_blocks 136336 00000020:00200000:3.0:1789760773.084727:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.084745:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.084786:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69791744, count: 4096 00000080:00200000:3.0:1789760773.084795:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69791744, bytes: 4096 00000080:00200000:3.0:1789760773.084811:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.084822:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.084824:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69791744, 69795840) 0 0 00000080:00200000:3.0:1789760773.084829:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.084845:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17039, 17039] 00020000:00200000:3.0:1789760773.084854:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69791744 chunk: [69791744, 69795840) 69795840, 4096 00020000:00200000:3.0:1789760773.084859:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.084882:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17039, 17039] 00000008:00010000:3.0:1789760773.084902:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.084909:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.084921:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69791744, 69795840) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.084936:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69791744=0x428f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.085007:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.085010:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69795840 00000020:00010000:3.0:1789760773.086503:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.086509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.086513:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.086517:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.086520:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.086533:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.086544:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.086588:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760773.086742:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 69791744<>69795840 00000080:00200000:3.0:1789760773.099063:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17039, 17039] 00000080:00200000:3.0:1789760773.099088:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.099094:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.099110:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69795840 i_blocks 136344 00000020:00200000:3.0:1789760773.099116:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.099130:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.099162:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69795840, count: 4096 00000080:00200000:3.0:1789760773.099169:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69795840, bytes: 4096 00000080:00200000:3.0:1789760773.099182:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.099190:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.099192:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69795840, 69799936) 0 0 00000080:00200000:3.0:1789760773.099197:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.099213:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17040, 17040] 00020000:00200000:3.0:1789760773.099222:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69795840 chunk: [69795840, 69799936) 69799936, 4096 00020000:00200000:3.0:1789760773.099227:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.099247:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17040, 17040] 00000008:00010000:3.0:1789760773.099265:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.099271:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.099284:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69795840, 69799936) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.099298:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69795840=0x4290000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.099367:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.099369:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69799936 00000020:00010000:3.0:1789760773.099378:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.099381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.099385:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.099388:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.099391:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.099400:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.099407:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.099429:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760773.099570:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 69795840<>69799936 00000080:00200000:3.0:1789760773.120861:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17040, 17040] 00000080:00200000:3.0:1789760773.120878:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.120885:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.120968:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69799936 i_blocks 136352 00000020:00200000:3.0:1789760773.120973:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.120983:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.121013:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69799936, count: 4096 00000080:00200000:3.0:1789760773.121020:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69799936, bytes: 4096 00000080:00200000:3.0:1789760773.121030:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.121039:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.121041:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69799936, 69804032) 0 0 00000080:00200000:3.0:1789760773.121046:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.121064:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17041, 17041] 00020000:00200000:3.0:1789760773.121073:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69799936 chunk: [69799936, 69804032) 69804032, 4096 00020000:00200000:3.0:1789760773.121078:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.121098:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17041, 17041] 00000008:00010000:3.0:1789760773.121118:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.121124:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.121135:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69799936, 69804032) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.121149:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69799936=0x4291000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.121206:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.121209:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69804032 00000020:00010000:3.0:1789760773.121216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.121384:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.121390:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.121393:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.121396:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.121406:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.121414:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.121432:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760773.121886:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 69799936<>69804032 00000080:00200000:3.0:1789760773.132749:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17041, 17041] 00000080:00200000:3.0:1789760773.132768:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.132775:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.132792:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69804032 i_blocks 136360 00000020:00200000:3.0:1789760773.132796:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.132806:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.132841:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69804032, count: 4096 00000080:00200000:3.0:1789760773.132848:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69804032, bytes: 4096 00000080:00200000:3.0:1789760773.132858:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.132866:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.132868:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69804032, 69808128) 0 0 00000080:00200000:3.0:1789760773.132873:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.132890:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17042, 17042] 00020000:00200000:3.0:1789760773.132900:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69804032 chunk: [69804032, 69808128) 69808128, 4096 00020000:00200000:3.0:1789760773.132906:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.132928:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17042, 17042] 00000008:00010000:3.0:1789760773.132946:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.132951:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.132962:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69804032, 69808128) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.132976:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69804032=0x4292000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.133034:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.133036:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69808128 00000020:00010000:3.0:1789760773.133044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.133047:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.133052:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.133055:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.133058:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.133066:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.133073:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.133089:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760773.133217:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 69804032<>69808128 00000080:00200000:3.0:1789760773.150790:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17042, 17042] 00000080:00200000:3.0:1789760773.150817:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.150825:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.150843:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69808128 i_blocks 136368 00000020:00200000:3.0:1789760773.150848:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.150863:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.150896:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69808128, count: 4096 00000080:00200000:3.0:1789760773.150903:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69808128, bytes: 4096 00000080:00200000:3.0:1789760773.150915:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.150924:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.150926:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69808128, 69812224) 0 0 00000080:00200000:3.0:1789760773.150930:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.150946:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17043, 17043] 00020000:00200000:3.0:1789760773.150955:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69808128 chunk: [69808128, 69812224) 69812224, 4096 00020000:00200000:3.0:1789760773.150959:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.150978:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17043, 17043] 00000008:00010000:3.0:1789760773.151000:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.151006:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.151016:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69808128, 69812224) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.151034:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69808128=0x4293000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.151102:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.151105:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69812224 00000020:00010000:3.0:1789760773.151113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.151117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.151122:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.151125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.151128:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.151137:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.151143:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.151176:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760773.151322:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 69808128<>69812224 00000100:00100000:3.0:1789760773.151369:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760773.151390:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565411200:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.170080:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17043, 17043] 00000080:00200000:1.0:1789760773.170099:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.170105:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.170122:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69812224 i_blocks 136376 00000020:00200000:1.0:1789760773.170126:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.170137:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.170171:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69812224, count: 4096 00000080:00200000:1.0:1789760773.170178:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69812224, bytes: 4096 00000080:00200000:1.0:1789760773.170191:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.170200:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.170202:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69812224, 69816320) 0 0 00000080:00200000:1.0:1789760773.170207:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.170225:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17044, 17044] 00020000:00200000:1.0:1789760773.170233:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69812224 chunk: [69812224, 69816320) 69816320, 4096 00020000:00200000:1.0:1789760773.170238:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.170258:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17044, 17044] 00000008:00010000:1.0:1789760773.170276:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.170282:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.170293:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69812224, 69816320) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.170307:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69812224=0x4294000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.170370:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.170373:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69816320 00000020:00010000:1.0:1789760773.170381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.170385:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.170388:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.170391:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.170394:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.170402:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.170409:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.170431:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760773.170573:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 69812224<>69816320 00000100:00100000:1.0:1789760773.186207:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984352 gen 3 00000008:00010000:1.0:1789760773.186358:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:1.0:1789760773.186366:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760773.186377:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760773.186400:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:1.0:1789760773.186430:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565411328:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.197856:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17045, 17045] 00000080:00200000:1.0:1789760773.197879:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.197886:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.197904:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69820416 i_blocks 136392 00000020:00200000:1.0:1789760773.197910:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.197925:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.197961:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69820416, count: 4096 00000080:00200000:1.0:1789760773.197969:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69820416, bytes: 4096 00000080:00200000:1.0:1789760773.197982:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.197991:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.197993:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69820416, 69824512) 0 0 00000080:00200000:1.0:1789760773.197997:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.198013:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17046, 17046] 00020000:00200000:1.0:1789760773.198022:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69820416 chunk: [69820416, 69824512) 69824512, 4096 00020000:00200000:1.0:1789760773.198026:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.198048:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17046, 17046] 00000008:00010000:1.0:1789760773.198065:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.198071:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.198083:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69820416, 69824512) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.198096:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69820416=0x4296000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.198165:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.198168:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69824512 00000020:00010000:1.0:1789760773.198177:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.198180:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.198183:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.198398:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.198403:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.198413:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.198420:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.198516:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760773.198786:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 69820416<>69824512 00000100:00100000:1.0:1789760773.213882:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984354 gen 3 00000080:00200000:1.0:1789760773.214384:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17046, 17046] 00000080:00200000:1.0:1789760773.214400:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.214407:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.214423:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69824512 i_blocks 136400 00000020:00200000:1.0:1789760773.214426:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.214436:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.214732:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69824512, count: 4096 00000080:00200000:1.0:1789760773.214740:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69824512, bytes: 4096 00000080:00200000:1.0:1789760773.214753:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.214762:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.214765:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69824512, 69828608) 0 0 00000080:00200000:1.0:1789760773.214770:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.214787:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17047, 17047] 00020000:00200000:1.0:1789760773.214796:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69824512 chunk: [69824512, 69828608) 69828608, 4096 00020000:00200000:1.0:1789760773.214801:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.214824:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17047, 17047] 00000008:00010000:1.0:1789760773.214842:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.214848:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.214859:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69824512, 69828608) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.214874:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69824512=0x4297000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.214936:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.214940:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69828608 00000020:00010000:1.0:1789760773.214949:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.214953:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.214956:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.214960:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.214963:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.214971:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.214978:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.214998:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760773.215138:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 69824512<>69828608 00000100:00100000:1.0:1789760773.231284:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984355 gen 3 00000008:00010000:1.0:1789760773.231335:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:1.0:1789760773.231345:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760773.231356:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760773.231384:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:1.0:1789760773.231415:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565411712:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.232391:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565411840:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.239925:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984356 gen 3 00000008:00010000:3.0:1789760773.239981:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760773.239989:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.239999:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.240025:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760773.240059:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565411840:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.242219:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000080:00200000:3.0:1789760773.262699:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17049, 17049] 00000080:00200000:3.0:1789760773.262722:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.262728:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.262745:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69836800 i_blocks 136424 00000020:00200000:3.0:1789760773.262750:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.262766:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.262801:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69836800, count: 4096 00000080:00200000:3.0:1789760773.262810:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69836800, bytes: 4096 00000080:00200000:3.0:1789760773.262823:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.262832:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.262835:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69836800, 69840896) 0 0 00000080:00200000:3.0:1789760773.262840:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.262856:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17050, 17050] 00020000:00200000:3.0:1789760773.262865:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69836800 chunk: [69836800, 69840896) 69840896, 4096 00020000:00200000:3.0:1789760773.262870:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.262891:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17050, 17050] 00000008:00010000:3.0:1789760773.262909:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.262916:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.263039:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69836800, 69840896) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.263056:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69836800=0x429a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.263111:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.263115:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69840896 00000020:00010000:3.0:1789760773.263123:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.263127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.263196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.263201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.263205:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.263215:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.263222:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.281749:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17050, 17050] 00000080:00200000:3.0:1789760773.281768:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.281775:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.281791:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69840896 i_blocks 136432 00000020:00200000:3.0:1789760773.281795:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.281805:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.281840:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69840896, count: 4096 00000080:00200000:3.0:1789760773.281847:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69840896, bytes: 4096 00000080:00200000:3.0:1789760773.281858:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.281867:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.281869:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69840896, 69844992) 0 0 00000080:00200000:3.0:1789760773.281873:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.281889:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17051, 17051] 00020000:00200000:3.0:1789760773.281909:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69840896 chunk: [69840896, 69844992) 69844992, 4096 00020000:00200000:3.0:1789760773.281915:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.281935:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17051, 17051] 00000008:00010000:3.0:1789760773.281954:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.281960:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.281972:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69840896, 69844992) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.281987:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69840896=0x429b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.282054:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.282057:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69844992 00000020:00010000:3.0:1789760773.282066:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.282069:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.282071:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.282075:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.282078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.282086:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.282092:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.282111:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760773.282236:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 69840896<>69844992 00000080:00200000:3.0:1789760773.292857:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17051, 17051] 00000080:00200000:3.0:1789760773.292874:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.292879:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.292893:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69844992 i_blocks 136440 00000020:00200000:3.0:1789760773.292960:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.292971:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.293005:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69844992, count: 4096 00000080:00200000:3.0:1789760773.293013:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69844992, bytes: 4096 00000080:00200000:3.0:1789760773.293025:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.293033:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.293036:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69844992, 69849088) 0 0 00000080:00200000:3.0:1789760773.293041:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.293059:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17052, 17052] 00020000:00200000:3.0:1789760773.293067:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69844992 chunk: [69844992, 69849088) 69849088, 4096 00020000:00200000:3.0:1789760773.293073:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.293093:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17052, 17052] 00000008:00010000:3.0:1789760773.293111:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.293117:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.293129:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69844992, 69849088) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.293142:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69844992=0x429c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.293199:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.293202:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69849088 00000020:00010000:3.0:1789760773.293210:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.293213:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.293216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.293220:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.293223:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.293231:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.293238:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.293254:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760773.293454:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 69844992<>69849088 00000080:00200000:3.0:1789760773.308224:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17052, 17052] 00000080:00200000:3.0:1789760773.308247:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.308254:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.308273:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69849088 i_blocks 136448 00000020:00200000:3.0:1789760773.308276:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.308289:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.308323:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69849088, count: 4096 00000080:00200000:3.0:1789760773.308331:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69849088, bytes: 4096 00000080:00200000:3.0:1789760773.308343:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.308352:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.308354:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69849088, 69853184) 0 0 00000080:00200000:3.0:1789760773.308358:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.308374:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17053, 17053] 00020000:00200000:3.0:1789760773.308382:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69849088 chunk: [69849088, 69853184) 69853184, 4096 00020000:00200000:3.0:1789760773.308386:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.308406:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17053, 17053] 00000008:00010000:3.0:1789760773.308425:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.308431:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.308442:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69849088, 69853184) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.308457:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69849088=0x429d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.308516:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.308518:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69853184 00000020:00010000:3.0:1789760773.308527:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.308530:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.308533:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.308537:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.308540:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.308559:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.308567:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.308592:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760773.308760:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 69849088<>69853184 00000100:00100000:3.0:1789760773.308803:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760773.308820:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565412480:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.318628:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17053, 17053] 00000080:00200000:1.0:1789760773.318676:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.318682:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.318698:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69853184 i_blocks 136456 00000020:00200000:1.0:1789760773.318702:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.318713:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.318743:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69853184, count: 4096 00000080:00200000:1.0:1789760773.318752:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69853184, bytes: 4096 00000080:00200000:1.0:1789760773.318763:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.318772:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.318775:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69853184, 69857280) 0 0 00000080:00200000:1.0:1789760773.318779:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.318797:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17054, 17054] 00020000:00200000:1.0:1789760773.318805:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69853184 chunk: [69853184, 69857280) 69857280, 4096 00020000:00200000:1.0:1789760773.318810:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.318829:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17054, 17054] 00000008:00010000:1.0:1789760773.318847:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.318852:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.318864:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69853184, 69857280) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.318877:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69853184=0x429e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.318939:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.318941:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69857280 00000020:00010000:1.0:1789760773.318949:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.318953:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.318956:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.318959:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.318963:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.318970:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.318976:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.318996:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760773.319141:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 69853184<>69857280 00000100:00100000:1.0:1789760773.319192:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760773.319217:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565412608:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760773.358478:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565412864:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.373314:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565412992:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.381842:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984365 gen 3 00000008:00010000:3.0:1789760773.381910:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760773.381920:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.382138:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.382163:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760773.382195:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565412992:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760773.383258:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565413120:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760773.392187:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17058, 17058] 00000080:00200000:3.0:1789760773.392203:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.392208:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.392222:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69873664 i_blocks 136496 00000020:00200000:3.0:1789760773.392225:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.392237:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.392271:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69873664, count: 4096 00000080:00200000:3.0:1789760773.392279:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69873664, bytes: 4096 00000080:00200000:3.0:1789760773.392289:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.392297:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.392300:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69873664, 69877760) 0 0 00000080:00200000:3.0:1789760773.392304:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.392321:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17059, 17059] 00020000:00200000:3.0:1789760773.392330:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69873664 chunk: [69873664, 69877760) 69877760, 4096 00020000:00200000:3.0:1789760773.392334:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.392354:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17059, 17059] 00000008:00010000:3.0:1789760773.392372:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.392378:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.392389:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69873664, 69877760) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.392405:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69873664=0x42a3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.392463:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.392467:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69877760 00000020:00010000:3.0:1789760773.392475:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.392478:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.392493:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.392497:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.392500:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.392508:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.392515:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.392532:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760773.392976:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 69873664<>69877760 00000100:00100000:3.0:1789760773.393026:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760773.393044:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565413248:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.405718:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984367 gen 3 00000008:00010000:3.0:1789760773.405778:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:3.0:1789760773.405787:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.405798:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.405826:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:3.0:1789760773.405857:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565413248:192.168.203.147@tcp:4:dd.0 00000020:00010000:0.0:1789760773.406836:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760773.406839:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.406842:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.406846:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760773.406848:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760773.406856:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760773.406863:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760773.407007:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760773.407142:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 69877760<>69881856 00000100:00100000:1.0:1789760773.407216:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565413376:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.417451:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984368 gen 3 00000008:00010000:0.0:1789760773.417576:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760773.417585:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.417597:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.417621:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000080:00200000:3.0:1789760773.417683:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17060, 17060] 00000100:00100000:0.0:1789760773.417684:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565413376:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760773.417702:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.417708:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.417723:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69881856 i_blocks 136512 00000020:00200000:3.0:1789760773.417727:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.417738:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.417772:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69881856, count: 4096 00000080:00200000:3.0:1789760773.417780:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69881856, bytes: 4096 00000080:00200000:3.0:1789760773.417790:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.417799:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.417801:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69881856, 69885952) 0 0 00000080:00200000:3.0:1789760773.417806:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.417821:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17061, 17061] 00020000:00200000:3.0:1789760773.417829:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69881856 chunk: [69881856, 69885952) 69885952, 4096 00020000:00200000:3.0:1789760773.417833:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.417851:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17061, 17061] 00000008:00010000:3.0:1789760773.417869:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.417875:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.417887:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69881856, 69885952) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.417902:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69881856=0x42a5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.418084:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.418087:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69885952 00000020:00010000:3.0:1789760773.418095:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.418099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.418103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.418106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.418108:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.418117:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.418123:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.418139:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760773.418262:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 69881856<>69885952 00000080:00200000:3.0:1789760773.436208:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17061, 17061] 00000080:00200000:3.0:1789760773.436226:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.436231:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.436257:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69885952 i_blocks 136520 00000020:00200000:3.0:1789760773.436261:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.436271:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.436304:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69885952, count: 4096 00000080:00200000:3.0:1789760773.436311:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69885952, bytes: 4096 00000080:00200000:3.0:1789760773.436321:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.436330:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.436333:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69885952, 69890048) 0 0 00000080:00200000:3.0:1789760773.436337:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.436355:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17062, 17062] 00020000:00200000:3.0:1789760773.436363:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69885952 chunk: [69885952, 69890048) 69890048, 4096 00020000:00200000:3.0:1789760773.436368:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.436388:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17062, 17062] 00000008:00010000:3.0:1789760773.436407:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.436414:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.436426:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69885952, 69890048) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.436439:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69885952=0x42a6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.436514:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.436518:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69890048 00000020:00010000:3.0:1789760773.436527:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.436531:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.436535:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.436539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.436541:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.436559:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.436566:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.436584:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760773.436735:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 69885952<>69890048 00000080:00200000:3.0:1789760773.452425:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17062, 17062] 00000080:00200000:3.0:1789760773.452441:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.452447:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.452462:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69890048 i_blocks 136528 00000020:00200000:3.0:1789760773.452465:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.452475:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.452529:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69890048, count: 4096 00000080:00200000:3.0:1789760773.452537:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69890048, bytes: 4096 00000080:00200000:3.0:1789760773.452557:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.452566:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.452569:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69890048, 69894144) 0 0 00000080:00200000:3.0:1789760773.452573:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.452590:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17063, 17063] 00020000:00200000:3.0:1789760773.452598:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69890048 chunk: [69890048, 69894144) 69894144, 4096 00020000:00200000:3.0:1789760773.452603:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.452622:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17063, 17063] 00000008:00010000:3.0:1789760773.452641:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.452692:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.452705:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69890048, 69894144) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.452718:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69890048=0x42a7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.452776:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.452779:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69894144 00000020:00010000:3.0:1789760773.452787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.452790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.452794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.452797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.452801:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.452810:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.452817:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.452834:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760773.453254:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 69890048<>69894144 00000100:00100000:3.0:1789760773.453301:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565413760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.466465:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984371 gen 3 00000008:00010000:3.0:1789760773.466518:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760773.466527:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.466538:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.466567:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760773.466595:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565413760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.468085:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565413888:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.480153:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984372 gen 3 00000008:00010000:3.0:1789760773.480205:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189900 00000020:00200000:3.0:1789760773.480213:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.480225:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.480256:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189900 00000100:00100000:3.0:1789760773.480289:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565413888:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.483494:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565414016:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.497266:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984373 gen 3 00000008:00010000:3.0:1789760773.497316:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189b40 00000020:00200000:3.0:1789760773.497324:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.497333:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.497361:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189b40 00000100:00100000:3.0:1789760773.497390:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565414016:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760773.509409:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17066, 17066] 00000080:00200000:3.0:1789760773.509426:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.509432:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.509447:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69906432 i_blocks 136560 00000020:00200000:3.0:1789760773.509451:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.509461:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.509493:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69906432, count: 4096 00000080:00200000:3.0:1789760773.509500:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69906432, bytes: 4096 00000080:00200000:3.0:1789760773.509510:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.509518:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.509521:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69906432, 69910528) 0 0 00000080:00200000:3.0:1789760773.509537:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.509565:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17067, 17067] 00020000:00200000:3.0:1789760773.509575:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69906432 chunk: [69906432, 69910528) 69910528, 4096 00020000:00200000:3.0:1789760773.509580:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.509711:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17067, 17067] 00000008:00010000:3.0:1789760773.509730:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.509736:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.509747:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69906432, 69910528) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.509760:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69906432=0x42ab000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.509819:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.509822:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69910528 00000020:00010000:3.0:1789760773.509831:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.509834:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.509837:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.509841:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.509844:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.509852:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.509859:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.509878:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760773.509999:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 69906432<>69910528 00000080:00200000:3.0:1789760773.525198:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17067, 17067] 00000080:00200000:3.0:1789760773.525215:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.525221:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.525237:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69910528 i_blocks 136568 00000020:00200000:3.0:1789760773.525240:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.525250:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.525282:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69910528, count: 4096 00000080:00200000:3.0:1789760773.525290:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69910528, bytes: 4096 00000080:00200000:3.0:1789760773.525303:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.525313:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.525315:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69910528, 69914624) 0 0 00000080:00200000:3.0:1789760773.525320:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.525339:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17068, 17068] 00020000:00200000:3.0:1789760773.525347:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69910528 chunk: [69910528, 69914624) 69914624, 4096 00020000:00200000:3.0:1789760773.525352:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.525373:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17068, 17068] 00000008:00010000:3.0:1789760773.525392:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.525398:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.525410:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69910528, 69914624) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.525430:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69910528=0x42ac000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.525860:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.525864:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69914624 00000020:00010000:3.0:1789760773.525874:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.525878:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.525882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.525886:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.525889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.525898:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.525906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.525929:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760773.526052:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 69910528<>69914624 00000080:00200000:3.0:1789760773.538393:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17068, 17068] 00000080:00200000:3.0:1789760773.538408:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.538415:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760773.539365:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565414528:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.550355:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984377 gen 3 00000008:00010000:3.0:1789760773.550622:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441890c0 00000020:00200000:3.0:1789760773.550630:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.550641:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.550691:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441890c0 00000100:00100000:3.0:1789760773.550722:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565414528:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760773.550755:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17069, 17069] 00000080:00200000:0.0:1789760773.550775:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760773.550783:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760773.550797:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69918720 i_blocks 136584 00000020:00200000:0.0:1789760773.550801:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760773.550811:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760773.550844:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69918720, count: 4096 00000080:00200000:0.0:1789760773.550851:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69918720, bytes: 4096 00000080:00200000:0.0:1789760773.550864:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760773.550872:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760773.550874:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69918720, 69922816) 0 0 00000080:00200000:0.0:1789760773.550879:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760773.550894:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17070, 17070] 00020000:00200000:0.0:1789760773.550903:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69918720 chunk: [69918720, 69922816) 69922816, 4096 00020000:00200000:0.0:1789760773.550907:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760773.550925:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17070, 17070] 00000008:00010000:0.0:1789760773.550944:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760773.550950:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760773.550961:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69918720, 69922816) append 0 append lockpos 0 00000080:00200000:0.0:1789760773.550975:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69918720=0x42ae000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760773.551365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760773.551369:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69922816 00000020:00010000:0.0:1789760773.551379:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760773.551383:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.551386:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.551390:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760773.551393:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760773.551400:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760773.551406:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760773.551425:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760773.551556:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 69918720<>69922816 00000100:00100000:0.0:1789760773.551610:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760773.551629:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565414656:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.562007:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984378 gen 3 00000008:00010000:0.0:1789760773.562058:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760773.562067:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.562078:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.562097:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760773.562135:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565414656:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.562301:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17070, 17070] 00000080:00200000:1.0:1789760773.562318:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.562325:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.563741:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69922816 i_blocks 136592 00000020:00200000:1.0:1789760773.563758:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.563778:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.563817:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69922816, count: 4096 00000080:00200000:1.0:1789760773.563826:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69922816, bytes: 4096 00000080:00200000:1.0:1789760773.563839:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.563849:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.563852:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69922816, 69926912) 0 0 00000080:00200000:1.0:1789760773.563858:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.563877:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17071, 17071] 00020000:00200000:1.0:1789760773.563889:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69922816 chunk: [69922816, 69926912) 69926912, 4096 00020000:00200000:1.0:1789760773.563894:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.563917:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17071, 17071] 00000008:00010000:1.0:1789760773.563938:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.563945:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.563958:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69922816, 69926912) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.563975:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69922816=0x42af000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.564046:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.564049:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69926912 00000020:00010000:1.0:1789760773.564058:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.564062:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.564065:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.564068:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0308 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.564071:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.564079:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.564087:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.564860:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760773.565272:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 69922816<>69926912 00000100:00100000:1.0:1789760773.565328:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760773.565353:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565414784:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.574794:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984379 gen 3 00000008:00010000:0.0:1789760773.574845:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760773.574854:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.574865:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.574893:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760773.574924:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565414784:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.575814:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565414912:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.586204:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984380 gen 3 00000008:00010000:3.0:1789760773.586252:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760773.586261:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.586271:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.586296:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760773.586327:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565414912:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.590849:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565415040:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760773.599440:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17073, 17073] 00000080:00200000:3.0:1789760773.599459:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.599466:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.599481:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69935104 i_blocks 136616 00000020:00200000:3.0:1789760773.599557:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.599569:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.599634:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69935104, count: 4096 00000080:00200000:3.0:1789760773.599641:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69935104, bytes: 4096 00000080:00200000:0.0:1789760773.617345:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17074, 17074] 00000080:00200000:0.0:1789760773.617361:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760773.617367:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760773.617380:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69939200 i_blocks 136624 00000020:00200000:0.0:1789760773.617384:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760773.617394:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760773.617425:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69939200, count: 4096 00000080:00200000:0.0:1789760773.617432:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69939200, bytes: 4096 00000080:00200000:0.0:1789760773.617444:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760773.617452:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760773.617454:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69939200, 69943296) 0 0 00000080:00200000:0.0:1789760773.617459:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760773.617475:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17075, 17075] 00020000:00200000:0.0:1789760773.617493:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69939200 chunk: [69939200, 69943296) 69943296, 4096 00020000:00200000:0.0:1789760773.617498:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760773.617515:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17075, 17075] 00000008:00010000:0.0:1789760773.617533:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760773.617539:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760773.617557:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69939200, 69943296) append 0 append lockpos 0 00000080:00200000:0.0:1789760773.617571:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69939200=0x42b3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760773.617626:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760773.617629:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69943296 00000020:00010000:0.0:1789760773.617636:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:0.0:1789760773.617640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.617678:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.617683:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760773.617686:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760773.617693:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760773.617700:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760773.617719:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760773.617847:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 69939200<>69943296 00000100:00100000:0.0:1789760773.617886:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760773.617914:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565415296:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.626176:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984383 gen 3 00000008:00010000:0.0:1789760773.626553:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760773.626563:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.626573:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.626692:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760773.626721:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565415296:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.629912:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760773.629951:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565415424:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.652811:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984384 gen 3 00000008:00010000:3.0:1789760773.652861:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760773.652869:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.652881:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.652910:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760773.652949:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565415424:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.653885:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565415552:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760773.667837:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760773.673329:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984385 gen 3 00000008:00010000:3.0:1789760773.673381:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760773.673390:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.673401:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.673428:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760773.673457:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565415552:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.674511:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565415680:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.691268:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984386 gen 3 00000008:00010000:3.0:1789760773.691324:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760773.691334:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.691347:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.691370:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760773.691402:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565415680:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.692605:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17078, 17078] 00000080:00200000:1.0:1789760773.692627:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.692633:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.692694:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69955584 i_blocks 136656 00000020:00200000:1.0:1789760773.692699:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.692734:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.692770:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69955584, count: 4096 00000080:00200000:1.0:1789760773.692778:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69955584, bytes: 4096 00000080:00200000:1.0:1789760773.692790:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.692800:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.692802:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69955584, 69959680) 0 0 00000080:00200000:1.0:1789760773.692806:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.692823:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17079, 17079] 00020000:00200000:1.0:1789760773.692832:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69955584 chunk: [69955584, 69959680) 69959680, 4096 00020000:00200000:1.0:1789760773.692837:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.692859:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17079, 17079] 00000008:00010000:1.0:1789760773.692876:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.692881:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.692967:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69955584, 69959680) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.692982:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69955584=0x42b7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.693039:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.693042:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69959680 00000020:00010000:1.0:1789760773.693050:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.693053:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.693056:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.693060:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.693063:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.693070:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.693317:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.693336:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760773.693468:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 69955584<>69959680 00000100:00100000:0.0:1789760773.693568:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565415808:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.706919:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984387 gen 3 00000008:00010000:0.0:1789760773.706970:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760773.706979:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.706989:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.707013:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760773.707042:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565415808:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760773.718507:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17080, 17080] 00000080:00200000:3.0:1789760773.718523:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.718529:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.718552:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69963776 i_blocks 136672 00000020:00200000:3.0:1789760773.718556:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.718567:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.718628:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69963776, count: 4096 00000080:00200000:3.0:1789760773.718637:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69963776, bytes: 4096 00000080:00200000:3.0:1789760773.718689:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.718698:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.718701:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69963776, 69967872) 0 0 00000080:00200000:3.0:1789760773.718706:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.718723:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17081, 17081] 00020000:00200000:3.0:1789760773.718732:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69963776 chunk: [69963776, 69967872) 69967872, 4096 00020000:00200000:3.0:1789760773.718737:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.718757:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17081, 17081] 00000008:00010000:3.0:1789760773.718775:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.718781:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.718793:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69963776, 69967872) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.718806:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69963776=0x42b9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.718860:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.718863:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69967872 00000020:00010000:3.0:1789760773.718871:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.718875:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.718878:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.718881:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.718884:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.718892:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.718898:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.718915:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760773.719036:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 69963776<>69967872 00000080:00200000:3.0:1789760773.730204:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17081, 17081] 00000080:00200000:3.0:1789760773.730220:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.730226:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.730242:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69967872 i_blocks 136680 00000020:00200000:3.0:1789760773.730246:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.730255:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.730290:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69967872, count: 4096 00000080:00200000:3.0:1789760773.730298:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69967872, bytes: 4096 00000080:00200000:3.0:1789760773.730309:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.730318:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.730321:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69967872, 69971968) 0 0 00000080:00200000:3.0:1789760773.730326:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.730343:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17082, 17082] 00020000:00200000:3.0:1789760773.730352:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69967872 chunk: [69967872, 69971968) 69971968, 4096 00020000:00200000:3.0:1789760773.730356:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.730376:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17082, 17082] 00000008:00010000:3.0:1789760773.730395:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.730401:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.730412:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69967872, 69971968) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.730426:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69967872=0x42ba000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.730485:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.730489:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69971968 00000020:00010000:3.0:1789760773.730496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.730500:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.730503:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.730507:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.730510:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.730518:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.730524:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.730557:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760773.731050:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 69967872<>69971968 00000080:00200000:1.0:1789760773.740341:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17082, 17082] 00000080:00200000:1.0:1789760773.740359:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.740365:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.740380:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69971968 i_blocks 136688 00000020:00200000:1.0:1789760773.740384:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.740393:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.740425:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69971968, count: 4096 00000080:00200000:1.0:1789760773.740433:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69971968, bytes: 4096 00000080:00200000:1.0:1789760773.740443:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.740452:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.740454:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69971968, 69976064) 0 0 00000080:00200000:1.0:1789760773.740459:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.740475:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17083, 17083] 00020000:00200000:1.0:1789760773.740598:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69971968 chunk: [69971968, 69976064) 69976064, 4096 00020000:00200000:1.0:1789760773.740604:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.740626:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17083, 17083] 00000008:00010000:1.0:1789760773.740738:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.740745:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.740759:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69971968, 69976064) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.740773:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69971968=0x42bb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.740833:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.740836:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69976064 00000020:00010000:1.0:1789760773.740844:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.740848:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.740850:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.740854:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.740856:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.740864:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.740871:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.740887:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760773.741019:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 69971968<>69976064 00000100:00100000:0.0:1789760773.741140:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565416320:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.751355:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984391 gen 3 00000008:00010000:0.0:1789760773.751407:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760773.751416:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.751426:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.751500:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760773.751534:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565416320:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.751966:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17083, 17083] 00000080:00200000:1.0:1789760773.751983:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.751989:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.752005:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69976064 i_blocks 136696 00000020:00200000:1.0:1789760773.752009:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.752019:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.752122:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69976064, count: 4096 00000080:00200000:1.0:1789760773.752131:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69976064, bytes: 4096 00000080:00200000:1.0:1789760773.752142:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.752151:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.752154:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69976064, 69980160) 0 0 00000080:00200000:1.0:1789760773.752159:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.752178:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17084, 17084] 00020000:00200000:1.0:1789760773.752187:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69976064 chunk: [69976064, 69980160) 69980160, 4096 00020000:00200000:1.0:1789760773.752191:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.752212:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17084, 17084] 00000008:00010000:1.0:1789760773.752229:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.752235:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.752248:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69976064, 69980160) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.752262:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69976064=0x42bc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.752327:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.752330:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69980160 00000020:00010000:1.0:1789760773.752339:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.752343:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.752346:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.752350:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.752354:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.752363:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.752370:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.752391:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760773.752991:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 69976064<>69980160 00000100:00100000:0.0:1789760773.753054:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760773.753188:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565416448:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.766238:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984392 gen 3 00000008:00010000:0.0:1789760773.766361:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760773.766370:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.766380:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.766410:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760773.766442:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565416448:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.769032:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565416576:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.783002:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984393 gen 3 00000008:00010000:3.0:1789760773.783445:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760773.783454:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760773.783465:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760773.783699:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000080:00200000:0.0:1789760773.783949:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17085, 17085] 00000080:00200000:0.0:1789760773.783969:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760773.783975:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760773.783992:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69984256 i_blocks 136712 00000020:00200000:0.0:1789760773.783997:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760773.784008:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000100:00100000:3.0:1789760773.784139:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565416576:192.168.203.147@tcp:4:dd.0 00000080:00200008:0.0:1789760773.784335:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69984256, count: 4096 00000080:00200000:0.0:1789760773.784344:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69984256, bytes: 4096 00000080:00200000:0.0:1789760773.784356:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760773.784366:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760773.784369:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69984256, 69988352) 0 0 00000080:00200000:0.0:1789760773.784374:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760773.784390:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17086, 17086] 00020000:00200000:0.0:1789760773.784400:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69984256 chunk: [69984256, 69988352) 69988352, 4096 00020000:00200000:0.0:1789760773.784405:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760773.784425:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17086, 17086] 00000008:00010000:0.0:1789760773.784443:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760773.784449:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760773.784461:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69984256, 69988352) append 0 append lockpos 0 00000080:00200000:0.0:1789760773.784476:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69984256=0x42be000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760773.784816:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760773.784820:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69988352 00000020:00010000:0.0:1789760773.784829:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:0.0:1789760773.784832:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.784836:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.784840:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760773.784843:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760773.784850:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760773.784857:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760773.784876:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760773.785121:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 69984256<>69988352 00000100:00100000:1.0:1789760773.785401:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565416704:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.795855:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984394 gen 3 00000008:00010000:0.0:1789760773.795917:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760773.795927:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.795938:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.796189:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760773.796220:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565416704:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760773.796639:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17086, 17086] 00000080:00200000:3.0:1789760773.796963:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.796970:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760773.797936:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565416832:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760773.807073:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17087, 17087] 00000080:00200000:0.0:1789760773.807090:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760773.807097:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760773.807113:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69992448 i_blocks 136728 00000020:00200000:0.0:1789760773.807117:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760773.807127:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760773.807162:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69992448, count: 4096 00000080:00200000:0.0:1789760773.807169:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69992448, bytes: 4096 00000080:00200000:0.0:1789760773.807179:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760773.807187:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760773.807189:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69992448, 69996544) 0 0 00000080:00200000:0.0:1789760773.807194:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760773.807212:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17088, 17088] 00020000:00200000:0.0:1789760773.807221:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69992448 chunk: [69992448, 69996544) 69996544, 4096 00020000:00200000:0.0:1789760773.807225:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760773.807244:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17088, 17088] 00000008:00010000:0.0:1789760773.807333:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760773.807341:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760773.807354:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69992448, 69996544) append 0 append lockpos 0 00000080:00200000:0.0:1789760773.807368:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69992448=0x42c0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760773.807424:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760773.807427:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 69996544 00000020:00010000:0.0:1789760773.807434:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:0.0:1789760773.807438:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.807441:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.807445:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760773.807447:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760773.807454:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760773.807460:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760773.807477:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760773.807983:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 69992448<>69996544 00000100:00100000:1.0:1789760773.808040:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760773.808120:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565416960:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.820306:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984396 gen 3 00000008:00010000:0.0:1789760773.820359:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760773.820368:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.820379:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.820441:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760773.820471:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565416960:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.820864:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17088, 17088] 00000080:00200000:1.0:1789760773.820882:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.820888:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.821098:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 69996544 i_blocks 136736 00000020:00200000:1.0:1789760773.821103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.821115:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.821151:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 69996544, count: 4096 00000080:00200000:1.0:1789760773.821158:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 69996544, bytes: 4096 00000080:00200000:1.0:1789760773.821171:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.821180:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.821183:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [69996544, 70000640) 0 0 00000080:00200000:1.0:1789760773.821187:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.821206:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17089, 17089] 00020000:00200000:1.0:1789760773.821215:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 69996544 chunk: [69996544, 70000640) 70000640, 4096 00020000:00200000:1.0:1789760773.821220:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.821240:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17089, 17089] 00000008:00010000:1.0:1789760773.821259:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.821266:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.821279:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [69996544, 70000640) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.821292:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=69996544=0x42c1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.821355:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.821358:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70000640 00000020:00010000:1.0:1789760773.821366:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.821370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.821373:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.821377:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0128 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.821380:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.821387:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.821394:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.821415:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760773.824288:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 69996544<>70000640 00000100:00100000:1.0:1789760773.824385:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760773.824415:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565417088:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.834965:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984397 gen 3 00000008:00010000:0.0:1789760773.835014:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760773.835022:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.835031:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.835124:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760773.835156:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565417088:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.838122:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000080:00200000:3.0:1789760773.850969:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17090, 17090] 00000080:00200000:3.0:1789760773.850985:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.850991:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.851009:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70004736 i_blocks 136752 00000020:00200000:3.0:1789760773.851013:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.851023:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.851090:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70004736, count: 4096 00000080:00200000:3.0:1789760773.851099:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70004736, bytes: 4096 00000080:00200000:3.0:1789760773.851110:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.851120:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.851122:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70004736, 70008832) 0 0 00000080:00200000:3.0:1789760773.851127:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.851144:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17091, 17091] 00020000:00200000:3.0:1789760773.851153:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70004736 chunk: [70004736, 70008832) 70008832, 4096 00020000:00200000:3.0:1789760773.851158:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.851178:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17091, 17091] 00000008:00010000:3.0:1789760773.851196:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.851202:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.851214:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70004736, 70008832) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.851228:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70004736=0x42c3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.851288:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.851291:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70008832 00000020:00010000:3.0:1789760773.851299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.851303:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.851306:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.851310:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.851313:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.851321:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.851328:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.851346:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760773.851468:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 70004736<>70008832 00000080:00200000:3.0:1789760773.865458:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17091, 17091] 00000080:00200000:3.0:1789760773.865475:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.865481:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.865507:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70008832 i_blocks 136760 00000020:00200000:3.0:1789760773.865511:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.865521:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.865563:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70008832, count: 4096 00000080:00200000:3.0:1789760773.865571:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70008832, bytes: 4096 00000080:00200000:3.0:1789760773.865581:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.865589:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.865591:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70008832, 70012928) 0 0 00000080:00200000:3.0:1789760773.865596:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.865611:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17092, 17092] 00020000:00200000:3.0:1789760773.865619:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70008832 chunk: [70008832, 70012928) 70012928, 4096 00020000:00200000:3.0:1789760773.865632:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.865695:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17092, 17092] 00000008:00010000:3.0:1789760773.865713:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.865718:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.865730:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70008832, 70012928) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.865744:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70008832=0x42c4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.865797:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.865800:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70012928 00000020:00010000:3.0:1789760773.865808:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.865812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.865815:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.865818:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.865821:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.865829:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.865835:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.865852:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760773.866196:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 70008832<>70012928 00000080:00200000:3.0:1789760773.877451:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17092, 17092] 00000080:00200000:3.0:1789760773.877467:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.877473:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760773.877500:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70012928 i_blocks 136768 00000020:00200000:3.0:1789760773.877504:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760773.877515:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760773.877553:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70012928, count: 4096 00000080:00200000:3.0:1789760773.877561:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70012928, bytes: 4096 00000080:00200000:3.0:1789760773.877570:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760773.877579:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760773.877582:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70012928, 70017024) 0 0 00000080:00200000:3.0:1789760773.877586:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760773.877602:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17093, 17093] 00020000:00200000:3.0:1789760773.877611:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70012928 chunk: [70012928, 70017024) 70017024, 4096 00020000:00200000:3.0:1789760773.877616:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760773.878285:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17093, 17093] 00000008:00010000:3.0:1789760773.878309:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760773.878315:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760773.878327:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70012928, 70017024) append 0 append lockpos 0 00000080:00200000:3.0:1789760773.878343:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70012928=0x42c5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760773.878407:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760773.878411:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70017024 00000020:00010000:3.0:1789760773.878419:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760773.878423:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.878426:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760773.878430:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760773.878433:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760773.878442:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760773.878449:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760773.878466:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760773.878611:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 70012928<>70017024 00000080:00200000:3.0:1789760773.887868:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17093, 17093] 00000080:00200000:3.0:1789760773.887883:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760773.887890:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760773.888640:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565417728:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760773.899960:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17094, 17094] 00000080:00200000:0.0:1789760773.899977:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760773.899984:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760773.900061:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70021120 i_blocks 136784 00000020:00200000:0.0:1789760773.900065:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760773.900076:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760773.900111:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70021120, count: 4096 00000080:00200000:0.0:1789760773.900119:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70021120, bytes: 4096 00000080:00200000:0.0:1789760773.900129:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760773.900138:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760773.900141:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70021120, 70025216) 0 0 00000080:00200000:0.0:1789760773.900146:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760773.900163:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17095, 17095] 00020000:00200000:0.0:1789760773.900172:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70021120 chunk: [70021120, 70025216) 70025216, 4096 00020000:00200000:0.0:1789760773.900177:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760773.900198:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17095, 17095] 00000008:00010000:0.0:1789760773.900343:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760773.900350:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760773.900362:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70021120, 70025216) append 0 append lockpos 0 00000080:00200000:0.0:1789760773.900376:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70021120=0x42c7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760773.900436:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760773.900439:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70025216 00000020:00010000:0.0:1789760773.900447:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760773.900451:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.900455:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.900458:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760773.900462:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760773.900468:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760773.900474:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760773.900491:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760773.900621:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 70021120<>70025216 00000100:00100000:1.0:1789760773.900768:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565417856:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.910033:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984403 gen 3 00000008:00010000:0.0:1789760773.910350:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760773.910361:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.910373:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.910437:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760773.910467:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565417856:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760773.912040:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565417984:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760773.923477:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17096, 17096] 00000080:00200000:0.0:1789760773.923583:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760773.923589:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760773.923604:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70029312 i_blocks 136800 00000020:00200000:0.0:1789760773.923607:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760773.923616:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760773.923673:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70029312, count: 4096 00000080:00200000:0.0:1789760773.923681:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70029312, bytes: 4096 00000080:00200000:0.0:1789760773.923691:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760773.923699:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760773.923702:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70029312, 70033408) 0 0 00000080:00200000:0.0:1789760773.923706:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760773.923722:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17097, 17097] 00020000:00200000:0.0:1789760773.923731:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70029312 chunk: [70029312, 70033408) 70033408, 4096 00020000:00200000:0.0:1789760773.923735:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760773.923755:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17097, 17097] 00000008:00010000:0.0:1789760773.923771:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760773.923777:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760773.923789:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70029312, 70033408) append 0 append lockpos 0 00000080:00200000:0.0:1789760773.923802:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70029312=0x42c9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760773.923859:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760773.923862:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70033408 00000020:00010000:0.0:1789760773.923871:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760773.923874:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.923877:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760773.923881:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760773.923885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760773.923893:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760773.923899:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760773.923916:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760773.924038:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 70029312<>70033408 00000100:00100000:0.0:1789760773.924091:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760773.924109:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565418112:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.937687:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984405 gen 3 00000008:00010000:0.0:1789760773.937736:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760773.937746:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.937756:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.937768:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760773.937798:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565418112:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.938122:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17097, 17097] 00000080:00200000:1.0:1789760773.938141:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.938148:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.938164:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70033408 i_blocks 136808 00000020:00200000:1.0:1789760773.938167:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.938181:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.938220:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70033408, count: 4096 00000080:00200000:1.0:1789760773.938229:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70033408, bytes: 4096 00000080:00200000:1.0:1789760773.938242:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.938252:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.938254:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70033408, 70037504) 0 0 00000080:00200000:1.0:1789760773.938259:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.938279:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17098, 17098] 00020000:00200000:1.0:1789760773.938290:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70033408 chunk: [70033408, 70037504) 70037504, 4096 00020000:00200000:1.0:1789760773.938296:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.938316:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17098, 17098] 00000008:00010000:1.0:1789760773.938335:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.938341:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.938353:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70033408, 70037504) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.938366:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70033408=0x42ca000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.938443:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.938446:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70037504 00000020:00010000:1.0:1789760773.938456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.938459:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.938463:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.938467:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.938470:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.938478:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.938485:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.938504:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760773.938962:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 70033408<>70037504 00000100:00100000:1.0:1789760773.939003:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760773.939024:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565418240:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.955074:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984406 gen 3 00000008:00010000:0.0:1789760773.955128:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760773.955136:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.955147:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.955222:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760773.955253:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565418240:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.956239:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17098, 17098] 00000080:00200000:1.0:1789760773.956259:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.956269:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.956286:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70037504 i_blocks 136816 00000020:00200000:1.0:1789760773.956290:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.956300:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.956336:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70037504, count: 4096 00000080:00200000:1.0:1789760773.956344:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70037504, bytes: 4096 00000080:00200000:1.0:1789760773.956356:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.956365:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.956367:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70037504, 70041600) 0 0 00000080:00200000:1.0:1789760773.956370:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.956388:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17099, 17099] 00020000:00200000:1.0:1789760773.956396:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70037504 chunk: [70037504, 70041600) 70041600, 4096 00020000:00200000:1.0:1789760773.956401:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.957990:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17099, 17099] 00000008:00010000:1.0:1789760773.958022:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.958029:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.958043:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70037504, 70041600) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.958060:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70037504=0x42cb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.958150:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.958153:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70041600 00000020:00010000:1.0:1789760773.958162:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.958170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.958173:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.958176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.958179:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.958191:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.958200:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.958233:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760773.958370:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 70037504<>70041600 00000100:00100000:0.0:1789760773.958880:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565418368:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.971701:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984407 gen 3 00000008:00010000:0.0:1789760773.971765:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760773.971774:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.971785:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.971797:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760773.971825:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565418368:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.972007:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17099, 17099] 00000080:00200000:1.0:1789760773.972024:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.972031:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.972047:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70041600 i_blocks 136824 00000020:00200000:1.0:1789760773.972051:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.972061:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.972095:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70041600, count: 4096 00000080:00200000:1.0:1789760773.972103:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70041600, bytes: 4096 00000080:00200000:1.0:1789760773.972113:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.972122:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.972124:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70041600, 70045696) 0 0 00000080:00200000:1.0:1789760773.972128:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.972145:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17100, 17100] 00020000:00200000:1.0:1789760773.972154:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70041600 chunk: [70041600, 70045696) 70045696, 4096 00020000:00200000:1.0:1789760773.972159:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.972178:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17100, 17100] 00000008:00010000:1.0:1789760773.972196:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.972202:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.972213:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70041600, 70045696) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.972229:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70041600=0x42cc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.972287:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.972291:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70045696 00000020:00010000:1.0:1789760773.972300:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.972303:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.972307:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.972310:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.972313:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.972321:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.972328:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.972407:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760773.972524:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 70041600<>70045696 00000100:00100000:0.0:1789760773.972718:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760773.972746:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565418496:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.982500:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984408 gen 3 00000008:00010000:0.0:1789760773.982562:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760773.982571:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.982581:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.982611:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000080:00200000:1.0:1789760773.982680:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17100, 17100] 00000100:00100000:0.0:1789760773.982683:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565418496:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760773.982700:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760773.982709:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760773.982725:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70045696 i_blocks 136832 00000020:00200000:1.0:1789760773.982730:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760773.982801:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760773.982834:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70045696, count: 4096 00000080:00200000:1.0:1789760773.982842:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70045696, bytes: 4096 00000080:00200000:1.0:1789760773.982851:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760773.982860:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760773.982862:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70045696, 70049792) 0 0 00000080:00200000:1.0:1789760773.982867:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760773.982883:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17101, 17101] 00020000:00200000:1.0:1789760773.983258:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70045696 chunk: [70045696, 70049792) 70049792, 4096 00020000:00200000:1.0:1789760773.983265:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760773.983287:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17101, 17101] 00000008:00010000:1.0:1789760773.983305:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760773.983311:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760773.983323:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70045696, 70049792) append 0 append lockpos 0 00000080:00200000:1.0:1789760773.983336:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70045696=0x42cd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760773.983396:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760773.983399:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70049792 00000020:00010000:1.0:1789760773.983407:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760773.983411:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.983414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760773.983418:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760773.983421:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760773.983428:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760773.983435:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760773.983466:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760773.983611:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 70045696<>70049792 00000100:00100000:0.0:1789760773.983716:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565418624:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760773.997223:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984409 gen 3 00000008:00010000:0.0:1789760773.997268:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760773.997278:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760773.997288:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760773.997380:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760773.997420:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565418624:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.011255:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984410 gen 3 00000008:00010000:3.0:1789760774.011311:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441890c0 00000020:00200000:3.0:1789760774.011320:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.011331:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.011391:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441890c0 00000080:00200000:1.0:1789760774.011592:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17102, 17102] 00000080:00200000:1.0:1789760774.011611:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000100:00100000:3.0:1789760774.011613:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565418752:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.011618:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.011632:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70053888 i_blocks 136848 00000020:00200000:1.0:1789760774.011636:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.011699:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.011734:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70053888, count: 4096 00000080:00200000:1.0:1789760774.011743:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70053888, bytes: 4096 00000080:00200000:1.0:1789760774.011753:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.011762:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.011765:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70053888, 70057984) 0 0 00000080:00200000:1.0:1789760774.011769:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.011787:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17103, 17103] 00020000:00200000:1.0:1789760774.011795:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70053888 chunk: [70053888, 70057984) 70057984, 4096 00020000:00200000:1.0:1789760774.011798:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.011816:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17103, 17103] 00000008:00010000:1.0:1789760774.011831:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.011836:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.011846:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70053888, 70057984) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.011859:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70053888=0x42cf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.011978:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.011981:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70057984 00000020:00010000:1.0:1789760774.011988:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.011991:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.011993:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.011996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.011998:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.012003:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.012008:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.012021:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760774.012190:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 70053888<>70057984 00000100:00100000:1.0:1789760774.012251:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565418880:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.027120:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984411 gen 3 00000008:00010000:0.0:1789760774.027173:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760774.027181:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.027192:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.027230:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760774.027263:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565418880:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.027418:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17103, 17103] 00000080:00200000:1.0:1789760774.027445:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.027452:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.027466:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70057984 i_blocks 136856 00000020:00200000:1.0:1789760774.027471:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.027481:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.027515:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70057984, count: 4096 00000080:00200000:1.0:1789760774.027523:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70057984, bytes: 4096 00000080:00200000:1.0:1789760774.027533:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.027552:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.027555:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70057984, 70062080) 0 0 00000080:00200000:1.0:1789760774.027559:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.027577:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17104, 17104] 00020000:00200000:1.0:1789760774.027586:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70057984 chunk: [70057984, 70062080) 70062080, 4096 00020000:00200000:1.0:1789760774.027590:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.027610:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17104, 17104] 00000008:00010000:1.0:1789760774.027629:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.027635:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.027695:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70057984, 70062080) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.027711:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70057984=0x42d0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.027773:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.027777:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70062080 00000020:00010000:1.0:1789760774.027785:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.027789:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.027791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.027795:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.027798:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.027806:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.027814:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.027832:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760774.027962:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 70057984<>70062080 00000100:00100000:0.0:1789760774.028127:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565419008:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.043567:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984412 gen 3 00000008:00010000:0.0:1789760774.043619:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760774.043628:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.043639:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.043688:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760774.043717:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565419008:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.043866:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17104, 17104] 00000080:00200000:1.0:1789760774.043881:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.043886:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.043902:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70062080 i_blocks 136864 00000020:00200000:1.0:1789760774.043906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.043915:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.043950:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70062080, count: 4096 00000080:00200000:1.0:1789760774.043957:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70062080, bytes: 4096 00000080:00200000:1.0:1789760774.043967:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.043976:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.043978:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70062080, 70066176) 0 0 00000080:00200000:1.0:1789760774.043983:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.043998:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17105, 17105] 00020000:00200000:1.0:1789760774.044006:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70062080 chunk: [70062080, 70066176) 70066176, 4096 00020000:00200000:1.0:1789760774.044011:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.044028:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17105, 17105] 00000008:00010000:1.0:1789760774.044045:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.044050:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.044061:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70062080, 70066176) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.044073:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70062080=0x42d1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.044125:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.044127:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70066176 00000020:00010000:1.0:1789760774.044133:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.044137:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.044140:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.044143:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.044146:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.044152:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.044158:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.044172:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760774.044504:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 70062080<>70066176 00000100:00100000:0.0:1789760774.044590:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565419136:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.059982:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984413 gen 3 00000008:00010000:0.0:1789760774.060030:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760774.060038:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.060049:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.060072:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760774.060101:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565419136:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.060334:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17105, 17105] 00000080:00200000:1.0:1789760774.060352:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.060358:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.060373:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70066176 i_blocks 136872 00000020:00200000:1.0:1789760774.060377:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.060387:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.060432:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70066176, count: 4096 00000080:00200000:1.0:1789760774.060449:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70066176, bytes: 4096 00000080:00200000:1.0:1789760774.060460:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.060469:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.060471:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70066176, 70070272) 0 0 00000080:00200000:1.0:1789760774.060475:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.060502:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17106, 17106] 00020000:00200000:1.0:1789760774.060511:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70066176 chunk: [70066176, 70070272) 70070272, 4096 00020000:00200000:1.0:1789760774.060515:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.060535:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17106, 17106] 00000008:00010000:1.0:1789760774.060563:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.060569:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.060581:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70066176, 70070272) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.060595:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70066176=0x42d2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.060696:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.060699:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70070272 00000020:00010000:1.0:1789760774.060707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.060711:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.060714:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.060717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.060721:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.060729:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.060736:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.060753:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760774.060877:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 70066176<>70070272 00000100:00100000:0.0:1789760774.063791:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565419264:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.080158:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984414 gen 3 00000008:00010000:0.0:1789760774.080208:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760774.080217:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.080229:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.080370:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760774.080401:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565419264:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.080798:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17106, 17106] 00000080:00200000:1.0:1789760774.080818:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.080824:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.080838:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70070272 i_blocks 136880 00000020:00200000:1.0:1789760774.080842:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.080852:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.080886:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70070272, count: 4096 00000080:00200000:1.0:1789760774.080894:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70070272, bytes: 4096 00000080:00200000:1.0:1789760774.080906:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.080915:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.080917:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70070272, 70074368) 0 0 00000080:00200000:1.0:1789760774.080921:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.080938:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17107, 17107] 00020000:00200000:1.0:1789760774.080947:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70070272 chunk: [70070272, 70074368) 70074368, 4096 00020000:00200000:1.0:1789760774.080952:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.080972:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17107, 17107] 00000008:00010000:1.0:1789760774.080990:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.080996:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.081009:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70070272, 70074368) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.081022:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70070272=0x42d3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.081080:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.081083:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70074368 00000020:00010000:1.0:1789760774.081092:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.081096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.081099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.081102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.081106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.081114:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.081122:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.081143:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760774.081267:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 70070272<>70074368 00000100:00100000:0.0:1789760774.081383:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760774.081406:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565419392:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.094930:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984415 gen 3 00000008:00010000:0.0:1789760774.094983:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760774.094991:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.095002:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.095023:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760774.095053:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565419392:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.095219:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17107, 17107] 00000080:00200000:1.0:1789760774.095236:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.095242:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.095256:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70074368 i_blocks 136888 00000020:00200000:1.0:1789760774.095260:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.095271:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.095315:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70074368, count: 4096 00000080:00200000:1.0:1789760774.095323:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70074368, bytes: 4096 00000080:00200000:1.0:1789760774.095333:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.095342:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.095345:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70074368, 70078464) 0 0 00000080:00200000:1.0:1789760774.095350:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.095368:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17108, 17108] 00020000:00200000:1.0:1789760774.095378:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70074368 chunk: [70074368, 70078464) 70078464, 4096 00020000:00200000:1.0:1789760774.095383:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.095403:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17108, 17108] 00000008:00010000:1.0:1789760774.095433:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.095450:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.095463:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70074368, 70078464) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.095478:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70074368=0x42d4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.095539:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.095558:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70078464 00000020:00010000:1.0:1789760774.095567:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.095570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.095574:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.095578:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.095581:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.095589:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.095596:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.095615:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760774.095797:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 70074368<>70078464 00000100:00100000:0.0:1789760774.096332:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565419520:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760774.107070:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984416 gen 3 00000008:00010000:1.0:1789760774.107116:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:1.0:1789760774.107124:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760774.107135:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760774.107157:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:1.0:1789760774.107185:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565419520:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760774.107595:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17108, 17108] 00000080:00200000:3.0:1789760774.107613:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760774.107620:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760774.107636:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70078464 i_blocks 136896 00000020:00200000:3.0:1789760774.107640:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760774.107673:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760774.107738:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70078464, count: 4096 00000080:00200000:3.0:1789760774.107748:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70078464, bytes: 4096 00000080:00200000:3.0:1789760774.107761:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760774.107771:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760774.107773:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70078464, 70082560) 0 0 00000080:00200000:3.0:1789760774.107778:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760774.107796:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17109, 17109] 00020000:00200000:3.0:1789760774.107805:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70078464 chunk: [70078464, 70082560) 70082560, 4096 00020000:00200000:3.0:1789760774.107810:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760774.107830:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17109, 17109] 00000008:00010000:3.0:1789760774.107849:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760774.107855:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760774.107867:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70078464, 70082560) append 0 append lockpos 0 00000080:00200000:3.0:1789760774.107881:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70078464=0x42d5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760774.107940:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.107943:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70082560 00000020:00010000:3.0:1789760774.107950:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760774.107954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.107958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.107962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760774.107965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760774.107974:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760774.107983:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760774.108000:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760774.108124:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 70078464<>70082560 00000100:00100000:3.0:1789760774.108172:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760774.108192:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565419648:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.120366:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984417 gen 3 00000008:00010000:3.0:1789760774.120418:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760774.120426:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.120437:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.120715:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760774.120754:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565419648:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.124432:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565419776:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.140343:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17110, 17110] 00000080:00200000:1.0:1789760774.140355:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.140360:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.140373:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70086656 i_blocks 136912 00000020:00200000:1.0:1789760774.140376:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.140384:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.140410:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70086656, count: 4096 00000080:00200000:1.0:1789760774.140416:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70086656, bytes: 4096 00000080:00200000:1.0:1789760774.140427:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.140435:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.140437:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70086656, 70090752) 0 0 00000080:00200000:1.0:1789760774.140441:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.140455:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17111, 17111] 00020000:00200000:1.0:1789760774.140461:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70086656 chunk: [70086656, 70090752) 70090752, 4096 00020000:00200000:1.0:1789760774.140465:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.140478:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17111, 17111] 00000008:00010000:1.0:1789760774.140491:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.140495:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.140504:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70086656, 70090752) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.140514:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70086656=0x42d7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.140708:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.140711:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70090752 00000020:00010000:1.0:1789760774.140718:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.140722:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.140725:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.140729:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.140732:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.140739:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.140748:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.140765:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760774.140922:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 70086656<>70090752 00000100:00100000:1.0:1789760774.140958:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760774.140974:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565419904:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.154976:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984419 gen 3 00000008:00010000:0.0:1789760774.155015:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760774.155089:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.155099:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.155180:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760774.155206:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565419904:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.156100:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565420032:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.173135:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17112, 17112] 00000080:00200000:0.0:1789760774.173147:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.173152:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.173163:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70094848 i_blocks 136928 00000020:00200000:0.0:1789760774.173166:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.173174:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.173199:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70094848, count: 4096 00000080:00200000:0.0:1789760774.173204:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70094848, bytes: 4096 00000080:00200000:0.0:1789760774.173221:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.173228:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.173230:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70094848, 70098944) 0 0 00000080:00200000:0.0:1789760774.173234:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.173246:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17113, 17113] 00020000:00200000:0.0:1789760774.173252:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70094848 chunk: [70094848, 70098944) 70098944, 4096 00020000:00200000:0.0:1789760774.173256:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.173268:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17113, 17113] 00000008:00010000:0.0:1789760774.173281:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.173286:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.173293:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70094848, 70098944) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.173302:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70094848=0x42d9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.173339:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.173341:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70098944 00000020:00010000:0.0:1789760774.173347:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.173350:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.173353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.173356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.173359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.173364:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.173369:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.173381:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760774.173479:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 70094848<>70098944 00000100:00100000:0.0:1789760774.173518:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760774.173531:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565420160:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.189883:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984421 gen 3 00000008:00010000:0.0:1789760774.189917:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760774.189924:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.189932:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.189960:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760774.189986:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565420160:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.204200:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17114, 17114] 00000080:00200000:0.0:1789760774.204223:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.204228:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.204240:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70103040 i_blocks 136944 00000020:00200000:0.0:1789760774.204243:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.204251:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.204281:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70103040, count: 4096 00000080:00200000:0.0:1789760774.204287:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70103040, bytes: 4096 00000080:00200000:0.0:1789760774.204298:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.204305:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.204306:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70103040, 70107136) 0 0 00000080:00200000:0.0:1789760774.204310:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.204322:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17115, 17115] 00020000:00200000:0.0:1789760774.204328:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70103040 chunk: [70103040, 70107136) 70107136, 4096 00020000:00200000:0.0:1789760774.204333:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.204348:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17115, 17115] 00000008:00010000:0.0:1789760774.204361:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.204376:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.204385:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70103040, 70107136) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.204396:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70103040=0x42db000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.204439:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.204442:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70107136 00000020:00010000:0.0:1789760774.204449:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.204452:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.204455:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.204458:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.204461:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.204466:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.204471:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.204484:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760774.204588:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 70103040<>70107136 00000100:00100000:1.0:1789760774.204828:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760774.204844:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565420416:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.222535:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984423 gen 3 00000008:00010000:0.0:1789760774.222584:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760774.222591:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.222599:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.222619:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760774.222641:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565420416:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.223570:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565420544:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.235505:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984424 gen 3 00000008:00010000:3.0:1789760774.235568:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760774.235577:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.235588:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.235610:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760774.235641:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565420544:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.236323:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760774.236343:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565420672:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.248143:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984425 gen 3 00000008:00010000:3.0:1789760774.248180:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441890c0 00000020:00200000:3.0:1789760774.248186:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.248194:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.248224:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441890c0 00000100:00100000:3.0:1789760774.248249:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565420672:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.250175:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760774.250201:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565420800:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.264092:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984426 gen 3 00000008:00010000:3.0:1789760774.264143:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760774.264151:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.264163:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.264190:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760774.264222:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565420800:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.265160:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565420928:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.273909:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17119, 17119] 00000080:00200000:0.0:1789760774.274029:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.274036:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.274050:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70123520 i_blocks 136984 00000020:00200000:0.0:1789760774.274052:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.274061:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.274092:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70123520, count: 4096 00000080:00200000:0.0:1789760774.274097:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70123520, bytes: 4096 00000080:00200000:0.0:1789760774.274109:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.274117:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.274120:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70123520, 70127616) 0 0 00000080:00200000:0.0:1789760774.274124:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.274141:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17120, 17120] 00020000:00200000:0.0:1789760774.274151:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70123520 chunk: [70123520, 70127616) 70127616, 4096 00020000:00200000:0.0:1789760774.274156:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.274176:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17120, 17120] 00000008:00010000:0.0:1789760774.274191:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.274196:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.274206:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70123520, 70127616) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.274217:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70123520=0x42e0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.274273:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.274276:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70127616 00000020:00010000:0.0:1789760774.274284:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.274287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.274290:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.274293:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.274295:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.274301:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.274307:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.274321:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760774.274435:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 70123520<>70127616 00000100:00100000:1.0:1789760774.274606:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565421056:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.292054:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984428 gen 3 00000008:00010000:0.0:1789760774.292104:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760774.292113:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.292123:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.292154:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760774.292185:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565421056:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.293713:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565421184:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.305835:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17121, 17121] 00000080:00200000:0.0:1789760774.305853:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.305859:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.305874:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70131712 i_blocks 137000 00000020:00200000:0.0:1789760774.305878:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.305898:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.305934:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70131712, count: 4096 00000080:00200000:0.0:1789760774.305941:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70131712, bytes: 4096 00000080:00200000:0.0:1789760774.305952:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.305961:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.305963:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70131712, 70135808) 0 0 00000080:00200000:0.0:1789760774.305968:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.305984:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17122, 17122] 00020000:00200000:0.0:1789760774.305993:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70131712 chunk: [70131712, 70135808) 70135808, 4096 00020000:00200000:0.0:1789760774.305997:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.306016:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17122, 17122] 00000008:00010000:0.0:1789760774.306031:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.306037:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.306047:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70131712, 70135808) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.306059:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70131712=0x42e2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.306113:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.306116:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70135808 00000020:00010000:0.0:1789760774.306125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.306128:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.306131:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.306134:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.306137:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.306143:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.306150:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.306165:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760774.306366:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 70131712<>70135808 00000100:00100000:0.0:1789760774.306419:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760774.306439:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565421312:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.317398:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984430 gen 3 00000008:00010000:0.0:1789760774.317442:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760774.317448:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.317457:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.317471:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760774.317509:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565421312:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.317834:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17122, 17122] 00000080:00200000:1.0:1789760774.317851:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.317856:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.317871:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70135808 i_blocks 137008 00000020:00200000:1.0:1789760774.317874:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.317955:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.317991:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70135808, count: 4096 00000080:00200000:1.0:1789760774.317999:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70135808, bytes: 4096 00000080:00200000:1.0:1789760774.318010:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.318019:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.318022:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70135808, 70139904) 0 0 00000080:00200000:1.0:1789760774.318026:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.318043:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17123, 17123] 00020000:00200000:1.0:1789760774.318051:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70135808 chunk: [70135808, 70139904) 70139904, 4096 00020000:00200000:1.0:1789760774.318055:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.318076:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17123, 17123] 00000008:00010000:1.0:1789760774.318093:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.318099:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.318110:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70135808, 70139904) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.318124:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70135808=0x42e3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.318184:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.318186:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70139904 00000020:00010000:1.0:1789760774.318195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.318198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.318202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.318205:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.318208:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.318215:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.318304:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.318322:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760774.318441:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 70135808<>70139904 00000100:00100000:0.0:1789760774.318607:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565421440:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.329881:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984431 gen 3 00000008:00010000:0.0:1789760774.329942:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760774.329951:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.329962:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.329986:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760774.330018:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565421440:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.330357:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17123, 17123] 00000080:00200000:1.0:1789760774.330374:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.330379:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.330395:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70139904 i_blocks 137016 00000020:00200000:1.0:1789760774.330399:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.330408:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.330443:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70139904, count: 4096 00000080:00200000:1.0:1789760774.330450:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70139904, bytes: 4096 00000080:00200000:1.0:1789760774.330461:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.330469:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.330472:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70139904, 70144000) 0 0 00000080:00200000:1.0:1789760774.330708:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.330729:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17124, 17124] 00020000:00200000:1.0:1789760774.330738:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70139904 chunk: [70139904, 70144000) 70144000, 4096 00020000:00200000:1.0:1789760774.330743:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.330764:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17124, 17124] 00000008:00010000:1.0:1789760774.330783:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.330790:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.330804:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70139904, 70144000) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.330818:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70139904=0x42e4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.331020:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.331024:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70144000 00000020:00010000:1.0:1789760774.331033:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.331037:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.331040:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.331044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.331047:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.331054:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.331061:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.331080:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760774.331207:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 70139904<>70144000 00000100:00100000:0.0:1789760774.331421:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565421568:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.339351:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984432 gen 3 00000008:00010000:0.0:1789760774.339403:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760774.339411:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.339423:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.339484:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760774.339515:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565421568:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.339798:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17124, 17124] 00000080:00200000:1.0:1789760774.339815:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.339821:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.339837:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70144000 i_blocks 137024 00000020:00200000:1.0:1789760774.339841:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.339853:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.339886:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70144000, count: 4096 00000080:00200000:1.0:1789760774.339894:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70144000, bytes: 4096 00000080:00200000:1.0:1789760774.339904:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.339913:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.339915:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70144000, 70148096) 0 0 00000080:00200000:1.0:1789760774.339920:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.340024:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17125, 17125] 00020000:00200000:1.0:1789760774.340034:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70144000 chunk: [70144000, 70148096) 70148096, 4096 00020000:00200000:1.0:1789760774.340038:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.340060:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17125, 17125] 00000008:00010000:1.0:1789760774.340080:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.340087:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.340354:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70144000, 70148096) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.340371:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70144000=0x42e5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.340450:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.340453:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70148096 00000020:00010000:1.0:1789760774.340463:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.340466:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.340469:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.340473:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.340476:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.340484:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.340491:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.340514:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760774.340677:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 70144000<>70148096 00000100:00100000:0.0:1789760774.340766:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565421696:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.353017:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984433 gen 3 00000008:00010000:0.0:1789760774.353076:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760774.353085:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.353096:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.353124:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760774.353155:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565421696:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.353349:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17125, 17125] 00000080:00200000:1.0:1789760774.353367:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.353373:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.353388:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70148096 i_blocks 137032 00000020:00200000:1.0:1789760774.353392:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.353401:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.353437:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70148096, count: 4096 00000080:00200000:1.0:1789760774.353444:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70148096, bytes: 4096 00000080:00200000:1.0:1789760774.353456:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.353463:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.353466:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70148096, 70152192) 0 0 00000080:00200000:1.0:1789760774.353471:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.353626:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17126, 17126] 00020000:00200000:1.0:1789760774.353637:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70148096 chunk: [70148096, 70152192) 70152192, 4096 00020000:00200000:1.0:1789760774.353688:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.353711:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17126, 17126] 00000008:00010000:1.0:1789760774.353730:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.353736:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.353748:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70148096, 70152192) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.353762:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70148096=0x42e6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.353822:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.353826:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70152192 00000020:00010000:1.0:1789760774.353834:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.353838:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.353841:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.353845:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.353848:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.353856:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.353863:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.353882:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760774.354152:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 70148096<>70152192 00000100:00100000:0.0:1789760774.354350:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565421824:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.365810:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984434 gen 3 00000008:00010000:0.0:1789760774.365860:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760774.365869:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.365881:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.365893:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760774.365923:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565421824:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.366038:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17126, 17126] 00000080:00200000:1.0:1789760774.366055:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.366062:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.366078:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70152192 i_blocks 137040 00000020:00200000:1.0:1789760774.366081:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.366092:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.366125:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70152192, count: 4096 00000080:00200000:1.0:1789760774.366133:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70152192, bytes: 4096 00000080:00200000:1.0:1789760774.366144:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.366153:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.366156:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70152192, 70156288) 0 0 00000080:00200000:1.0:1789760774.366160:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.366179:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17127, 17127] 00020000:00200000:1.0:1789760774.366188:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70152192 chunk: [70152192, 70156288) 70156288, 4096 00020000:00200000:1.0:1789760774.366203:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.366223:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17127, 17127] 00000008:00010000:1.0:1789760774.366242:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.366248:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.366382:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70152192, 70156288) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.366397:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70152192=0x42e7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.366466:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.366469:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70156288 00000020:00010000:1.0:1789760774.366478:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.366481:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.366484:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.366488:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.366491:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.366499:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.366507:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.366523:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760774.366694:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 70152192<>70156288 00000100:00100000:0.0:1789760774.366989:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760774.367009:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565421952:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.381591:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984435 gen 3 00000008:00010000:0.0:1789760774.381905:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760774.381915:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.381925:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.381979:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760774.382007:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565421952:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.382082:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17127, 17127] 00000080:00200000:1.0:1789760774.382100:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.382106:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.382122:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70156288 i_blocks 137048 00000020:00200000:1.0:1789760774.382127:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.382136:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.382170:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70156288, count: 4096 00000080:00200000:1.0:1789760774.382178:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70156288, bytes: 4096 00000080:00200000:1.0:1789760774.382189:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.382198:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.382200:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70156288, 70160384) 0 0 00000080:00200000:1.0:1789760774.382206:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.382222:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17128, 17128] 00020000:00200000:1.0:1789760774.382232:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70156288 chunk: [70156288, 70160384) 70160384, 4096 00020000:00200000:1.0:1789760774.382237:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.382256:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17128, 17128] 00000008:00010000:1.0:1789760774.382273:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.382280:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.382292:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70156288, 70160384) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.382306:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70156288=0x42e8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.382363:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.382366:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70160384 00000020:00010000:1.0:1789760774.382374:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.382378:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.382381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.382385:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.382388:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.382394:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.382401:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.382416:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760774.382605:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 70156288<>70160384 00000100:00100000:0.0:1789760774.382841:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565422080:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.402519:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984436 gen 3 00000008:00010000:0.0:1789760774.402584:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760774.402593:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.402603:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.402744:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760774.402778:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565422080:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.402862:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17128, 17128] 00000080:00200000:1.0:1789760774.402878:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.402946:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.402964:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70160384 i_blocks 137056 00000020:00200000:1.0:1789760774.402967:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.402978:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.403012:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70160384, count: 4096 00000080:00200000:1.0:1789760774.403020:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70160384, bytes: 4096 00000080:00200000:1.0:1789760774.403035:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.403043:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.403045:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70160384, 70164480) 0 0 00000080:00200000:1.0:1789760774.403050:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.403068:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17129, 17129] 00020000:00200000:1.0:1789760774.403077:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70160384 chunk: [70160384, 70164480) 70164480, 4096 00020000:00200000:1.0:1789760774.403082:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.403102:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17129, 17129] 00000008:00010000:1.0:1789760774.403123:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.403130:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.403140:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70160384, 70164480) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.403154:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70160384=0x42e9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.403213:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.403216:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70164480 00000020:00010000:1.0:1789760774.403224:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.403228:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.403237:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.403241:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.403244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.403252:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.403259:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.403276:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760774.403406:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 70160384<>70164480 00000100:00100000:0.0:1789760774.403832:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565422208:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.418387:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984437 gen 3 00000008:00010000:0.0:1789760774.418439:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760774.418448:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.418460:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.418527:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760774.418568:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565422208:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760774.419203:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17129, 17129] 00000080:00200000:1.0:1789760774.419230:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760774.419236:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.419254:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70164480 i_blocks 137064 00000020:00200000:1.0:1789760774.419258:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.419268:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.419303:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70164480, count: 4096 00000080:00200000:1.0:1789760774.419311:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70164480, bytes: 4096 00000080:00200000:1.0:1789760774.419323:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.419332:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.419334:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70164480, 70168576) 0 0 00000080:00200000:1.0:1789760774.419339:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.419356:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17130, 17130] 00020000:00200000:1.0:1789760774.419365:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70164480 chunk: [70164480, 70168576) 70168576, 4096 00020000:00200000:1.0:1789760774.419369:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.419390:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17130, 17130] 00000008:00010000:1.0:1789760774.420387:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.420397:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.420423:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70164480, 70168576) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.420441:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70164480=0x42ea000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.420769:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.420773:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70168576 00000020:00010000:1.0:1789760774.420783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.420787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.420790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.420793:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.420797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.420808:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.420816:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.420863:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760774.421758:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 70164480<>70168576 00000100:00100000:1.0:1789760774.421841:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565422336:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.434269:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984438 gen 3 00000008:00010000:0.0:1789760774.434321:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760774.434330:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.434342:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.434434:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760774.434465:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565422336:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.437701:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565422464:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.455322:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984439 gen 3 00000008:00010000:3.0:1789760774.455379:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760774.455388:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.455399:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.455548:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760774.455592:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565422464:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.457033:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565422592:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.472745:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17132, 17132] 00000080:00200000:0.0:1789760774.472765:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.472772:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.472789:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70176768 i_blocks 137088 00000020:00200000:0.0:1789760774.472793:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.472804:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.472839:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70176768, count: 4096 00000080:00200000:0.0:1789760774.472849:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70176768, bytes: 4096 00000080:00200000:0.0:1789760774.472859:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.472868:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.472871:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70176768, 70180864) 0 0 00000080:00200000:0.0:1789760774.472876:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.472893:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17133, 17133] 00020000:00200000:0.0:1789760774.472914:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70176768 chunk: [70176768, 70180864) 70180864, 4096 00020000:00200000:0.0:1789760774.472920:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.472942:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17133, 17133] 00000008:00010000:0.0:1789760774.472961:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.472968:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.472980:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70176768, 70180864) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.472998:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70176768=0x42ed000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.479128:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.479141:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70180864 00000020:00010000:0.0:1789760774.479160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.479165:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.479170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.479174:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.479176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.479191:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.479201:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.479246:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760774.479395:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 70176768<>70180864 00000100:00100000:1.0:1789760774.479486:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565422720:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760774.490214:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984441 gen 3 00000008:00010000:0.0:1789760774.490267:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760774.490276:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760774.490287:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760774.490314:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760774.490346:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565422720:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.492325:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565422848:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.500531:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984442 gen 3 00000008:00010000:3.0:1789760774.500595:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760774.500604:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.500616:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.500686:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760774.500718:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565422848:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.501761:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565422976:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.515331:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984443 gen 3 00000008:00010000:3.0:1789760774.515382:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760774.515391:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.515403:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.515566:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760774.515599:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565422976:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.516503:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565423104:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.539617:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17136, 17136] 00000080:00200000:0.0:1789760774.539634:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.539639:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760774.540555:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565423232:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.554469:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984445 gen 3 00000008:00010000:3.0:1789760774.554579:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760774.554588:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.554599:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.554622:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760774.554964:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565423232:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.556825:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565423360:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.577551:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984446 gen 3 00000008:00010000:3.0:1789760774.577598:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760774.577607:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.577619:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.577642:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760774.577826:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565423360:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.579354:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760774.579380:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565423488:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.603363:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984447 gen 3 00000008:00010000:3.0:1789760774.603411:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760774.603419:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.603429:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.603452:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760774.603486:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565423488:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.605151:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760774.605174:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565423616:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.618068:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984448 gen 3 00000008:00010000:3.0:1789760774.618119:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760774.618128:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.618295:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.618374:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760774.618403:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565423616:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.620130:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760774.620153:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565423744:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.634573:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984449 gen 3 00000008:00010000:3.0:1789760774.634624:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441890c0 00000020:00200000:3.0:1789760774.634633:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.634689:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.634713:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441890c0 00000100:00100000:3.0:1789760774.634899:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565423744:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.636574:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760774.636602:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565423872:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.648281:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984450 gen 3 00000008:00010000:3.0:1789760774.648333:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760774.648343:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.648356:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.648384:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760774.648415:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565423872:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.649989:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565424000:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760774.663343:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984451 gen 3 00000008:00010000:3.0:1789760774.663406:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760774.663414:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760774.663425:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760774.663523:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760774.663566:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565424000:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760774.680695:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17144, 17144] 00000080:00200000:3.0:1789760774.680712:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760774.680718:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760774.680734:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70225920 i_blocks 137184 00000020:00200000:3.0:1789760774.680738:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760774.680749:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760774.680785:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70225920, count: 4096 00000080:00200000:3.0:1789760774.680792:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70225920, bytes: 4096 00000080:00200000:3.0:1789760774.680803:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760774.680811:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760774.680814:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70225920, 70230016) 0 0 00000080:00200000:3.0:1789760774.680818:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760774.680836:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17145, 17145] 00020000:00200000:3.0:1789760774.680845:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70225920 chunk: [70225920, 70230016) 70230016, 4096 00020000:00200000:3.0:1789760774.680852:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760774.680872:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17145, 17145] 00000008:00010000:3.0:1789760774.680957:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760774.680964:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760774.680976:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70225920, 70230016) append 0 append lockpos 0 00000080:00200000:3.0:1789760774.680991:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70225920=0x42f9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760774.681053:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.681057:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70230016 00000020:00010000:3.0:1789760774.681065:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760774.681069:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.681072:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.681076:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760774.681079:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760774.681087:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760774.681094:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760774.681110:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760774.681305:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 70225920<>70230016 00010000:00010000:3.0:1789760774.683899:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1789760774.691285:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17145, 17145] 00000080:00200000:3.0:1789760774.691306:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760774.691313:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760774.691328:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70230016 i_blocks 137192 00000020:00200000:3.0:1789760774.691332:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760774.691345:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760774.691382:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70230016, count: 4096 00000080:00200000:3.0:1789760774.691389:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70230016, bytes: 4096 00000080:00200000:3.0:1789760774.691398:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760774.691407:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760774.691409:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70230016, 70234112) 0 0 00000080:00200000:3.0:1789760774.691413:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760774.691433:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17146, 17146] 00020000:00200000:3.0:1789760774.691443:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70230016 chunk: [70230016, 70234112) 70234112, 4096 00020000:00200000:3.0:1789760774.691448:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760774.691468:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17146, 17146] 00000008:00010000:3.0:1789760774.691503:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760774.691510:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760774.691522:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70230016, 70234112) append 0 append lockpos 0 00000080:00200000:3.0:1789760774.691546:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70230016=0x42fa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760774.691608:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.691611:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70234112 00000020:00010000:3.0:1789760774.691621:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760774.691624:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.691627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.691630:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760774.691680:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760774.691689:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760774.691696:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760774.691712:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760774.691831:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 70230016<>70234112 00000080:00200000:3.0:1789760774.702742:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17146, 17146] 00000080:00200000:3.0:1789760774.702759:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760774.702765:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760774.702781:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70234112 i_blocks 137200 00000020:00200000:3.0:1789760774.702784:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760774.702793:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760774.702828:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70234112, count: 4096 00000080:00200000:3.0:1789760774.702835:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70234112, bytes: 4096 00000080:00200000:3.0:1789760774.702845:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760774.702853:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760774.702856:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70234112, 70238208) 0 0 00000080:00200000:3.0:1789760774.702860:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760774.702876:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17147, 17147] 00020000:00200000:3.0:1789760774.702896:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70234112 chunk: [70234112, 70238208) 70238208, 4096 00020000:00200000:3.0:1789760774.702900:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760774.702920:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17147, 17147] 00000008:00010000:3.0:1789760774.702949:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760774.702956:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760774.702967:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70234112, 70238208) append 0 append lockpos 0 00000080:00200000:3.0:1789760774.702981:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70234112=0x42fb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760774.703037:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.703040:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70238208 00000020:00010000:3.0:1789760774.703049:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760774.703053:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.703055:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.703059:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760774.703062:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760774.703070:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760774.703077:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760774.703094:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760774.703229:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 70234112<>70238208 00000080:00200000:3.0:1789760774.716402:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17147, 17147] 00000080:00200000:3.0:1789760774.716419:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760774.716425:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760774.716440:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70238208 i_blocks 137208 00000020:00200000:3.0:1789760774.716444:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760774.716455:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760774.716499:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70238208, count: 4096 00000080:00200000:3.0:1789760774.716506:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70238208, bytes: 4096 00000080:00200000:3.0:1789760774.716516:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760774.716525:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760774.716527:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70238208, 70242304) 0 0 00000080:00200000:3.0:1789760774.716531:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760774.716621:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17148, 17148] 00020000:00200000:3.0:1789760774.716630:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70238208 chunk: [70238208, 70242304) 70242304, 4096 00020000:00200000:3.0:1789760774.716635:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760774.716730:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17148, 17148] 00000008:00010000:3.0:1789760774.716747:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760774.716753:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760774.716765:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70238208, 70242304) append 0 append lockpos 0 00000080:00200000:3.0:1789760774.716778:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70238208=0x42fc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760774.716831:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.716834:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70242304 00000020:00010000:3.0:1789760774.716842:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760774.716845:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.716848:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.716852:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760774.716855:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760774.716862:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760774.716867:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760774.716883:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760774.716996:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 70238208<>70242304 00000080:00200000:3.0:1789760774.728982:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17148, 17148] 00000080:00200000:3.0:1789760774.728999:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760774.729006:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760774.729020:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70242304 i_blocks 137216 00000020:00200000:3.0:1789760774.729023:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760774.729032:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760774.729064:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70242304, count: 4096 00000080:00200000:3.0:1789760774.729070:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70242304, bytes: 4096 00000080:00200000:3.0:1789760774.729080:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760774.729087:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760774.729090:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70242304, 70246400) 0 0 00000080:00200000:3.0:1789760774.729094:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760774.729110:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17149, 17149] 00020000:00200000:3.0:1789760774.729119:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70242304 chunk: [70242304, 70246400) 70246400, 4096 00020000:00200000:3.0:1789760774.729123:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760774.729141:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17149, 17149] 00000008:00010000:3.0:1789760774.729158:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760774.729164:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760774.729572:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70242304, 70246400) append 0 append lockpos 0 00000080:00200000:3.0:1789760774.729588:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70242304=0x42fd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760774.729712:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.729715:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70246400 00000020:00010000:3.0:1789760774.729723:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760774.729727:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.729730:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.729733:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760774.729736:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760774.729744:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760774.729751:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760774.730066:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760774.730187:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 70242304<>70246400 00000080:00200000:3.0:1789760774.744344:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17149, 17149] 00000080:00200000:3.0:1789760774.744365:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760774.744371:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760774.744386:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70246400 i_blocks 137224 00000020:00200000:3.0:1789760774.744390:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760774.744401:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760774.744428:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70246400, count: 4096 00000080:00200000:3.0:1789760774.744434:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70246400, bytes: 4096 00000080:00200000:3.0:1789760774.744442:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760774.744449:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760774.744452:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70246400, 70250496) 0 0 00000080:00200000:3.0:1789760774.744455:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760774.744468:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17150, 17150] 00020000:00200000:3.0:1789760774.744487:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70246400 chunk: [70246400, 70250496) 70250496, 4096 00020000:00200000:3.0:1789760774.744491:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760774.744508:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17150, 17150] 00000008:00010000:3.0:1789760774.744523:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760774.744530:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760774.744549:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70246400, 70250496) append 0 append lockpos 0 00000080:00200000:3.0:1789760774.744560:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70246400=0x42fe000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760774.744612:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.744615:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70250496 00000020:00010000:3.0:1789760774.744621:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760774.744625:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.744628:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.744631:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760774.744634:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760774.744641:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760774.745262:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760774.745284:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760774.745389:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 70246400<>70250496 00000080:00200000:3.0:1789760774.765750:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17150, 17150] 00000080:00200000:3.0:1789760774.765763:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760774.765768:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760774.765781:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70250496 i_blocks 137232 00000020:00200000:3.0:1789760774.765785:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760774.765792:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760774.765817:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70250496, count: 4096 00000080:00200000:3.0:1789760774.765824:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70250496, bytes: 4096 00000080:00200000:3.0:1789760774.765831:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760774.765838:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760774.765840:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70250496, 70254592) 0 0 00000080:00200000:3.0:1789760774.765844:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760774.765857:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17151, 17151] 00020000:00200000:3.0:1789760774.765864:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70250496 chunk: [70250496, 70254592) 70254592, 4096 00020000:00200000:3.0:1789760774.765868:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760774.765883:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17151, 17151] 00000008:00010000:3.0:1789760774.765896:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760774.765912:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760774.765922:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70250496, 70254592) append 0 append lockpos 0 00000080:00200000:3.0:1789760774.765933:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70250496=0x42ff000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760774.765972:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.765974:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70254592 00000020:00010000:3.0:1789760774.765982:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760774.765985:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.765987:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.765991:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760774.765993:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760774.765999:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760774.766005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.784073:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17151, 17151] 00000080:00200000:3.0:1789760774.784091:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760774.784097:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760774.784112:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70254592 i_blocks 137240 00000020:00200000:3.0:1789760774.784116:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760774.784128:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760774.784154:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70254592, count: 4096 00000080:00200000:3.0:1789760774.784161:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70254592, bytes: 4096 00000080:00200000:3.0:1789760774.784172:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760774.784181:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760774.784183:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70254592, 70258688) 0 0 00000080:00200000:3.0:1789760774.784187:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760774.784198:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17152, 17152] 00020000:00200000:3.0:1789760774.784205:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70254592 chunk: [70254592, 70258688) 70258688, 4096 00020000:00200000:3.0:1789760774.784209:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760774.784225:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17152, 17152] 00000008:00010000:3.0:1789760774.784238:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760774.784242:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760774.784251:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70254592, 70258688) append 0 append lockpos 0 00000080:00200000:3.0:1789760774.784261:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70254592=0x4300000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760774.784313:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.784316:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70258688 00000020:00010000:3.0:1789760774.784323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760774.784326:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.784329:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.784332:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760774.784335:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760774.784342:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760774.784347:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760774.784370:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760774.784466:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 70254592<>70258688 00000080:00200000:3.0:1789760774.808014:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17152, 17152] 00000080:00200000:3.0:1789760774.808035:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760774.808041:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760774.808054:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70258688 i_blocks 137248 00000020:00200000:3.0:1789760774.808059:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760774.808070:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760774.808095:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70258688, count: 4096 00000080:00200000:3.0:1789760774.808102:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70258688, bytes: 4096 00000080:00200000:3.0:1789760774.808113:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760774.808121:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760774.808123:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70258688, 70262784) 0 0 00000080:00200000:3.0:1789760774.808126:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760774.808138:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17153, 17153] 00020000:00200000:3.0:1789760774.808145:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70258688 chunk: [70258688, 70262784) 70262784, 4096 00020000:00200000:3.0:1789760774.808149:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760774.808166:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17153, 17153] 00000008:00010000:3.0:1789760774.808180:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760774.808184:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760774.808192:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70258688, 70262784) append 0 append lockpos 0 00000080:00200000:3.0:1789760774.808203:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70258688=0x4301000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760774.808249:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760774.808252:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70262784 00000020:00010000:3.0:1789760774.808259:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760774.808549:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.808553:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760774.808557:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760774.808560:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760774.808568:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760774.808575:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760774.808594:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760774.809188:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66d80 - object 0x240000400:5 offset 70258688<>70262784 00000080:00200000:0.0:1789760774.833165:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17153, 17153] 00000080:00200000:0.0:1789760774.833186:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.833192:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.833232:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70262784 i_blocks 137256 00000020:00200000:0.0:1789760774.833237:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.833247:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.833273:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70262784, count: 4096 00000080:00200000:0.0:1789760774.833280:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70262784, bytes: 4096 00000080:00200000:0.0:1789760774.833290:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.833298:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.833300:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70262784, 70266880) 0 0 00000080:00200000:0.0:1789760774.833303:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.833315:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17154, 17154] 00020000:00200000:0.0:1789760774.833322:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70262784 chunk: [70262784, 70266880) 70266880, 4096 00020000:00200000:0.0:1789760774.833326:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.833341:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17154, 17154] 00000008:00010000:0.0:1789760774.833356:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.833360:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.833369:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70262784, 70266880) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.833380:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70262784=0x4302000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.833425:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.833427:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70266880 00000020:00010000:0.0:1789760774.833434:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.833437:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.833440:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.833443:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.833446:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.833452:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.834085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.834106:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760774.834215:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 70262784<>70266880 00000100:00100000:0.0:1789760774.834256:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760774.834275:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565425408:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760774.860318:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984462 gen 3 00000008:00010000:1.0:1789760774.860358:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760774.860365:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760774.860374:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760774.860393:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760774.860416:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565425408:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.860505:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17154, 17154] 00000080:00200000:0.0:1789760774.860517:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.860522:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.860532:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70266880 i_blocks 137264 00000020:00200000:0.0:1789760774.860545:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.860554:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.860579:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70266880, count: 4096 00000080:00200000:0.0:1789760774.860585:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70266880, bytes: 4096 00000080:00200000:0.0:1789760774.860592:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.860600:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.860602:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70266880, 70270976) 0 0 00000080:00200000:0.0:1789760774.860606:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.860619:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17155, 17155] 00020000:00200000:0.0:1789760774.860626:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70266880 chunk: [70266880, 70270976) 70270976, 4096 00020000:00200000:0.0:1789760774.860629:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.860675:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17155, 17155] 00000008:00010000:0.0:1789760774.860688:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.860693:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.860701:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70266880, 70270976) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.860711:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70266880=0x4303000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.860752:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.860755:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70270976 00000020:00010000:0.0:1789760774.860761:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.860764:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.860767:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.860770:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.860773:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.860779:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.860785:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.860799:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760774.860911:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 70266880<>70270976 00000100:00100000:1.0:1789760774.861427:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760774.861443:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565425536:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760774.879283:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984463 gen 3 00000008:00010000:1.0:1789760774.879320:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760774.879326:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760774.879334:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760774.879396:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:1.0:1789760774.879421:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565425536:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.879942:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17155, 17155] 00000080:00200000:0.0:1789760774.879955:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.879960:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.879982:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70270976 i_blocks 137272 00000020:00200000:0.0:1789760774.879985:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.880002:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.880029:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70270976, count: 4096 00000080:00200000:0.0:1789760774.880035:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70270976, bytes: 4096 00000080:00200000:0.0:1789760774.880044:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.880051:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.880053:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70270976, 70275072) 0 0 00000080:00200000:0.0:1789760774.880058:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.880071:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17156, 17156] 00020000:00200000:0.0:1789760774.880078:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70270976 chunk: [70270976, 70275072) 70275072, 4096 00020000:00200000:0.0:1789760774.880081:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.880106:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17156, 17156] 00000008:00010000:0.0:1789760774.880120:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.880124:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.880133:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70270976, 70275072) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.880144:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70270976=0x4304000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.880870:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.880873:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70275072 00000020:00010000:0.0:1789760774.880881:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.880885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.880887:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.880890:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.880893:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.880900:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.880906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.880964:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760774.881389:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 70270976<>70275072 00000100:00100000:1.0:1789760774.881560:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565425664:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760774.895635:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984464 gen 3 00000008:00010000:1.0:1789760774.895701:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760774.895708:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760774.895717:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760774.895736:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760774.895761:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565425664:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.895863:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17156, 17156] 00000080:00200000:0.0:1789760774.895876:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.895881:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.895895:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70275072 i_blocks 137280 00000020:00200000:0.0:1789760774.895898:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.895907:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.895940:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70275072, count: 4096 00000080:00200000:0.0:1789760774.895947:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70275072, bytes: 4096 00000080:00200000:0.0:1789760774.895956:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.895963:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.895965:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70275072, 70279168) 0 0 00000080:00200000:0.0:1789760774.895969:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.895990:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17157, 17157] 00020000:00200000:0.0:1789760774.896004:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70275072 chunk: [70275072, 70279168) 70279168, 4096 00020000:00200000:0.0:1789760774.896008:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.896024:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17157, 17157] 00000008:00010000:0.0:1789760774.896038:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.896043:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.896052:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70275072, 70279168) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.896062:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70275072=0x4305000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.896103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.896105:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70279168 00000020:00010000:0.0:1789760774.899688:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.899693:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.899697:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.899701:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.899703:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.899715:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.899724:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.899758:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760774.899884:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70275072<>70279168 00000100:00100000:1.0:1789760774.900038:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760774.900066:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565425792:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760774.909813:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984465 gen 3 00000008:00010000:1.0:1789760774.909851:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760774.909857:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760774.909864:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760774.909940:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:1.0:1789760774.909965:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565425792:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.911350:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17157, 17157] 00000080:00200000:0.0:1789760774.911366:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.911372:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760774.911474:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70279168 i_blocks 137288 00000020:00200000:1.0:1789760774.911479:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760774.911491:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760774.911517:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70279168, count: 4096 00000080:00200000:1.0:1789760774.911524:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70279168, bytes: 4096 00000080:00200000:1.0:1789760774.911533:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760774.911549:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760774.911552:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70279168, 70283264) 0 0 00000080:00200000:1.0:1789760774.911557:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760774.911569:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17158, 17158] 00020000:00200000:1.0:1789760774.911576:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70279168 chunk: [70279168, 70283264) 70283264, 4096 00020000:00200000:1.0:1789760774.911580:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760774.911596:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17158, 17158] 00000008:00010000:1.0:1789760774.911610:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760774.911614:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760774.911623:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70279168, 70283264) append 0 append lockpos 0 00000080:00200000:1.0:1789760774.911634:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70279168=0x4306000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760774.911706:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760774.911709:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70283264 00000020:00010000:1.0:1789760774.911717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760774.911720:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.911722:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760774.911725:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760774.911729:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760774.911736:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760774.911741:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760774.911757:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760774.911861:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 70279168<>70283264 00000100:00100000:0.0:1789760774.912173:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565425920:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760774.922126:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984466 gen 3 00000008:00010000:1.0:1789760774.922165:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760774.922171:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760774.922179:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760774.922197:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760774.922220:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565425920:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.922694:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17158, 17158] 00000080:00200000:0.0:1789760774.922708:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.922713:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.922726:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70283264 i_blocks 137296 00000020:00200000:0.0:1789760774.922729:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.922738:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.922763:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70283264, count: 4096 00000080:00200000:0.0:1789760774.922768:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70283264, bytes: 4096 00000080:00200000:0.0:1789760774.922777:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.922785:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.922787:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70283264, 70287360) 0 0 00000080:00200000:0.0:1789760774.922791:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.922806:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17159, 17159] 00020000:00200000:0.0:1789760774.922813:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70283264 chunk: [70283264, 70287360) 70287360, 4096 00020000:00200000:0.0:1789760774.922817:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.922834:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17159, 17159] 00000008:00010000:0.0:1789760774.922849:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.922854:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.922864:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70283264, 70287360) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.922873:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70283264=0x4307000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.922982:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.922985:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70287360 00000020:00010000:0.0:1789760774.922993:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.923063:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.923067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.923070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.923073:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.923081:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.923087:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.923101:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:0.0:1789760774.923193:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 70283264<>70287360 00000100:00100000:1.0:1789760774.923257:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565426048:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760774.939712:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984467 gen 3 00000008:00010000:1.0:1789760774.939748:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760774.939754:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760774.939763:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760774.939786:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760774.939811:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565426048:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.939912:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17159, 17159] 00000080:00200000:0.0:1789760774.939925:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.939930:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.939941:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70287360 i_blocks 137304 00000020:00200000:0.0:1789760774.939944:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.939954:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.939982:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70287360, count: 4096 00000080:00200000:0.0:1789760774.939988:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70287360, bytes: 4096 00000080:00200000:0.0:1789760774.940007:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.940015:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.940017:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70287360, 70291456) 0 0 00000080:00200000:0.0:1789760774.940022:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.940036:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17160, 17160] 00020000:00200000:0.0:1789760774.940044:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70287360 chunk: [70287360, 70291456) 70291456, 4096 00020000:00200000:0.0:1789760774.940047:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.940063:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17160, 17160] 00000008:00010000:0.0:1789760774.940076:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.940081:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.940090:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70287360, 70291456) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.940101:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70287360=0x4308000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.940144:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.940146:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70291456 00000020:00010000:0.0:1789760774.940403:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.940407:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.940410:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.940413:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.940415:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.940422:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.940428:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.940456:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:0.0:1789760774.940709:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41f80 - object 0x240000400:5 offset 70287360<>70291456 00000100:00100000:1.0:1789760774.940775:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760774.940788:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565426176:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760774.949819:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984468 gen 3 00000008:00010000:1.0:1789760774.949857:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760774.949863:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760774.949871:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760774.949889:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760774.949913:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565426176:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.949995:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17160, 17160] 00000080:00200000:0.0:1789760774.950010:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.950014:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.950025:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70291456 i_blocks 137312 00000020:00200000:0.0:1789760774.950028:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.950037:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.950062:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70291456, count: 4096 00000080:00200000:0.0:1789760774.950067:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70291456, bytes: 4096 00000080:00200000:0.0:1789760774.950073:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.950080:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.950082:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70291456, 70295552) 0 0 00000080:00200000:0.0:1789760774.950086:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.950100:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17161, 17161] 00020000:00200000:0.0:1789760774.950107:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70291456 chunk: [70291456, 70295552) 70295552, 4096 00020000:00200000:0.0:1789760774.950111:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.950125:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17161, 17161] 00000008:00010000:0.0:1789760774.950138:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.950143:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.950236:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70291456, 70295552) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.950248:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70291456=0x4309000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.950292:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.950295:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70295552 00000020:00010000:0.0:1789760774.950302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.950305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.950308:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.950310:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.950313:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.950319:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.950325:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.950338:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760774.950436:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40700 - object 0x240000400:5 offset 70291456<>70295552 00000100:00100000:1.0:1789760774.950849:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760774.950865:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565426304:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760774.966850:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984469 gen 3 00000008:00010000:1.0:1789760774.966886:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760774.966892:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760774.966900:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760774.966921:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:1.0:1789760774.966946:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565426304:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.967073:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17161, 17161] 00000080:00200000:0.0:1789760774.967086:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.967091:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.967103:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70295552 i_blocks 137320 00000020:00200000:0.0:1789760774.967106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.967114:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.967140:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70295552, count: 4096 00000080:00200000:0.0:1789760774.967145:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70295552, bytes: 4096 00000080:00200000:0.0:1789760774.967154:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.967161:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.967163:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70295552, 70299648) 0 0 00000080:00200000:0.0:1789760774.967167:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.967179:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17162, 17162] 00020000:00200000:0.0:1789760774.967186:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70295552 chunk: [70295552, 70299648) 70299648, 4096 00020000:00200000:0.0:1789760774.967190:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.967206:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17162, 17162] 00000008:00010000:0.0:1789760774.967220:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.967225:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.967234:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70295552, 70299648) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.967243:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70295552=0x430a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.967282:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.967285:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70299648 00000020:00010000:0.0:1789760774.967290:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.967294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.967296:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.967299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.967301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.967307:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.967312:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.967325:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760774.967446:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43100 - object 0x240000400:5 offset 70295552<>70299648 00000100:00100000:1.0:1789760774.967624:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760774.967639:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565426432:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760774.984398:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984470 gen 3 00000008:00010000:1.0:1789760774.984434:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760774.984440:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760774.985108:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760774.985129:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760774.985240:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565426432:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760774.985341:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17162, 17162] 00000080:00200000:0.0:1789760774.985353:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760774.985358:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760774.985370:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70299648 i_blocks 137328 00000020:00200000:0.0:1789760774.985373:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760774.985383:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760774.985412:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70299648, count: 4096 00000080:00200000:0.0:1789760774.985417:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70299648, bytes: 4096 00000080:00200000:0.0:1789760774.985425:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760774.985432:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760774.985434:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70299648, 70303744) 0 0 00000080:00200000:0.0:1789760774.985437:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760774.985521:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17163, 17163] 00020000:00200000:0.0:1789760774.985528:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70299648 chunk: [70299648, 70303744) 70303744, 4096 00020000:00200000:0.0:1789760774.985532:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760774.985556:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17163, 17163] 00000008:00010000:0.0:1789760774.985570:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760774.985575:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760774.985584:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70299648, 70303744) append 0 append lockpos 0 00000080:00200000:0.0:1789760774.985594:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70299648=0x430b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760774.985637:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760774.985639:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70303744 00000020:00010000:0.0:1789760774.986073:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760774.986078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.986081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760774.986085:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760774.986087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760774.986095:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760774.986101:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760774.986118:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760774.987163:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70299648<>70303744 00000100:00100000:1.0:1789760774.987253:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565426560:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760775.002842:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984471 gen 3 00000008:00010000:0.0:1789760775.002879:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.002886:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.002894:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.002916:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.002939:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565426560:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760775.003021:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17163, 17163] 00000080:00200000:3.0:1789760775.003034:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.003038:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.003049:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70303744 i_blocks 137336 00000020:00200000:3.0:1789760775.003052:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.003061:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.003085:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70303744, count: 4096 00000080:00200000:3.0:1789760775.003091:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70303744, bytes: 4096 00000080:00200000:3.0:1789760775.003099:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.003106:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.003108:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70303744, 70307840) 0 0 00000080:00200000:3.0:1789760775.003113:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.003126:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17164, 17164] 00020000:00200000:3.0:1789760775.003134:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70303744 chunk: [70303744, 70307840) 70307840, 4096 00020000:00200000:3.0:1789760775.003137:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.003152:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17164, 17164] 00000008:00010000:3.0:1789760775.003166:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.003171:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.003180:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70303744, 70307840) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.003193:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70303744=0x430c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.003236:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.003239:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70307840 00000020:00010000:3.0:1789760775.003245:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.003248:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.003250:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.003253:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.003256:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.003262:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.003267:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.003280:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760775.003368:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 70303744<>70307840 00000080:00200000:3.0:1789760775.014340:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17164, 17164] 00000080:00200000:3.0:1789760775.014353:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.014358:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.014370:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70307840 i_blocks 137344 00000020:00200000:3.0:1789760775.014373:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.014380:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.014405:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70307840, count: 4096 00000080:00200000:3.0:1789760775.014411:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70307840, bytes: 4096 00000080:00200000:3.0:1789760775.014419:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.014427:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.014429:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70307840, 70311936) 0 0 00000080:00200000:3.0:1789760775.014433:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.014445:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17165, 17165] 00020000:00200000:3.0:1789760775.014452:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70307840 chunk: [70307840, 70311936) 70311936, 4096 00020000:00200000:3.0:1789760775.014455:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.014470:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17165, 17165] 00000008:00010000:3.0:1789760775.014494:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.014498:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.014506:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70307840, 70311936) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.014516:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70307840=0x430d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.014565:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.014568:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70311936 00000020:00010000:3.0:1789760775.014583:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.014587:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.014590:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.014593:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.014596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.014601:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.014606:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.014619:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760775.014953:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 70307840<>70311936 00000100:00100000:3.0:1789760775.014984:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760775.014999:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565426816:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.033094:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17165, 17165] 00000080:00200000:0.0:1789760775.033106:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.033111:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.033123:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70311936 i_blocks 137352 00000020:00200000:0.0:1789760775.033127:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.033135:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.033160:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70311936, count: 4096 00000080:00200000:0.0:1789760775.033166:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70311936, bytes: 4096 00000080:00200000:0.0:1789760775.033174:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.033181:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.033183:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70311936, 70316032) 0 0 00000080:00200000:0.0:1789760775.033186:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.033198:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17166, 17166] 00020000:00200000:0.0:1789760775.033204:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70311936 chunk: [70311936, 70316032) 70316032, 4096 00020000:00200000:0.0:1789760775.033208:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.033222:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17166, 17166] 00000008:00010000:0.0:1789760775.033235:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.033240:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.033249:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70311936, 70316032) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.033259:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70311936=0x430e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.033302:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.033305:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70316032 00000020:00010000:0.0:1789760775.033311:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.033314:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.033316:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.033319:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.033322:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.033327:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.033332:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.033345:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.033432:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70311936<>70316032 00000100:00100000:1.0:1789760775.033495:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565426944:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760775.052513:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984474 gen 3 00000008:00010000:0.0:1789760775.052563:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.052569:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.052578:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.052681:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.052703:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565426944:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.054245:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565427072:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.068124:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984475 gen 3 00000008:00010000:3.0:1789760775.068171:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441890c0 00000020:00200000:3.0:1789760775.068178:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760775.068185:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760775.068204:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441890c0 00000100:00100000:3.0:1789760775.068242:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565427072:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.068820:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565427200:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.085452:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984476 gen 3 00000008:00010000:3.0:1789760775.085489:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760775.085495:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760775.085504:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760775.085692:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760775.085717:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565427200:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.086342:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565427328:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760775.101884:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17169, 17169] 00000080:00200000:3.0:1789760775.101897:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.101902:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.101913:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70328320 i_blocks 137384 00000020:00200000:3.0:1789760775.101917:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.101925:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.101951:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70328320, count: 4096 00000080:00200000:3.0:1789760775.101958:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70328320, bytes: 4096 00000080:00200000:3.0:1789760775.101967:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.101975:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.101977:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70328320, 70332416) 0 0 00000080:00200000:3.0:1789760775.101981:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.101993:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17170, 17170] 00020000:00200000:3.0:1789760775.102000:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70328320 chunk: [70328320, 70332416) 70332416, 4096 00020000:00200000:3.0:1789760775.102004:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.102017:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17170, 17170] 00000008:00010000:3.0:1789760775.102030:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.102035:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.102044:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70328320, 70332416) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.102054:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70328320=0x4312000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.102092:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.102095:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70332416 00000020:00010000:3.0:1789760775.102101:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.102104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.102108:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.102110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.102113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.102120:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.102125:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:3.0:1789760775.111597:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760775.111914:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565427456:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.125482:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984478 gen 3 00000008:00010000:3.0:1789760775.125516:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760775.125523:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760775.125541:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760775.125562:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760775.125586:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565427456:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760775.144150:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17171, 17171] 00000080:00200000:3.0:1789760775.144173:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.144180:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.144197:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70336512 i_blocks 137400 00000020:00200000:3.0:1789760775.144202:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.144215:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.144254:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70336512, count: 4096 00000080:00200000:3.0:1789760775.144262:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70336512, bytes: 4096 00000080:00200000:3.0:1789760775.144274:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.144284:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.144286:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70336512, 70340608) 0 0 00000080:00200000:3.0:1789760775.144291:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.144308:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17172, 17172] 00020000:00200000:3.0:1789760775.144317:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70336512 chunk: [70336512, 70340608) 70340608, 4096 00020000:00200000:3.0:1789760775.144322:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.144342:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17172, 17172] 00000008:00010000:3.0:1789760775.144360:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.144368:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.144380:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70336512, 70340608) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.144395:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70336512=0x4314000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.144461:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.144464:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70340608 00000020:00010000:3.0:1789760775.144484:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.144488:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.144491:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a5a8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.144495:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.144498:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.144507:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.144513:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.144542:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760775.144725:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 70336512<>70340608 00000100:00100000:3.0:1789760775.144843:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760775.144875:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565427712:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.158137:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984480 gen 3 00000008:00010000:3.0:1789760775.158188:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760775.158197:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760775.158208:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760775.158300:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760775.158333:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565427712:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.159194:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565427840:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.171265:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984481 gen 3 00000008:00010000:3.0:1789760775.171321:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760775.171331:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760775.171342:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760775.171362:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760775.171393:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565427840:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760775.171623:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17173, 17173] 00000080:00200000:1.0:1789760775.171755:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760775.171762:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.172091:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70344704 i_blocks 137416 00000020:00200000:0.0:1789760775.172097:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.172107:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.172142:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70344704, count: 4096 00000080:00200000:0.0:1789760775.172149:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70344704, bytes: 4096 00000080:00200000:0.0:1789760775.172160:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.172168:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.172170:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70344704, 70348800) 0 0 00000080:00200000:0.0:1789760775.172175:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.172192:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17174, 17174] 00020000:00200000:0.0:1789760775.172202:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70344704 chunk: [70344704, 70348800) 70348800, 4096 00020000:00200000:0.0:1789760775.172206:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.172230:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17174, 17174] 00000008:00010000:0.0:1789760775.172248:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.172253:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.172266:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70344704, 70348800) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.172279:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70344704=0x4316000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.172342:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.172345:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70348800 00000020:00010000:0.0:1789760775.172353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.172356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.172359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.172363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.172365:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.172374:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.172381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.172398:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.172723:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70344704<>70348800 00000100:00100000:1.0:1789760775.172800:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565427968:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760775.182012:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984482 gen 3 00000008:00010000:0.0:1789760775.182060:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.182072:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.182085:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.182108:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.182137:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565427968:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760775.182217:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17174, 17174] 00000080:00200000:3.0:1789760775.182232:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.182236:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.182248:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70348800 i_blocks 137424 00000020:00200000:3.0:1789760775.182251:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.182261:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.182290:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70348800, count: 4096 00000080:00200000:3.0:1789760775.182296:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70348800, bytes: 4096 00000080:00200000:3.0:1789760775.182304:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.182310:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.182313:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70348800, 70352896) 0 0 00000080:00200000:3.0:1789760775.182316:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.182329:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17175, 17175] 00020000:00200000:3.0:1789760775.182336:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70348800 chunk: [70348800, 70352896) 70352896, 4096 00020000:00200000:3.0:1789760775.182339:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.182354:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17175, 17175] 00000008:00010000:3.0:1789760775.182370:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.182375:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.182384:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70348800, 70352896) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.182395:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70348800=0x4317000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.182443:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.182446:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70352896 00000020:00010000:3.0:1789760775.182454:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.182457:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.182460:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.182464:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.182467:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.182483:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.182491:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.182508:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760775.182640:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 70348800<>70352896 00000100:00100000:3.0:1789760775.182727:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760775.182743:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565428096:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.195906:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984483 gen 3 00000008:00010000:3.0:1789760775.195960:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760775.195968:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760775.195980:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760775.195997:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760775.196027:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565428096:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.196210:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17175, 17175] 00000080:00200000:0.0:1789760775.196228:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.196234:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.196250:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70352896 i_blocks 137432 00000020:00200000:0.0:1789760775.196254:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.196265:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.196300:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70352896, count: 4096 00000080:00200000:0.0:1789760775.196308:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70352896, bytes: 4096 00000080:00200000:0.0:1789760775.196320:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.196329:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.196333:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70352896, 70356992) 0 0 00000080:00200000:0.0:1789760775.197058:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.197081:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17176, 17176] 00020000:00200000:0.0:1789760775.197092:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70352896 chunk: [70352896, 70356992) 70356992, 4096 00020000:00200000:0.0:1789760775.197457:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.197483:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17176, 17176] 00000008:00010000:0.0:1789760775.197504:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.197510:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.197523:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70352896, 70356992) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.197553:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70352896=0x4318000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.197629:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.197632:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70356992 00000020:00010000:0.0:1789760775.197641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.199145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.199151:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.199156:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.199159:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.199173:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.199185:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.200389:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.200563:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70352896<>70356992 00000100:00100000:0.0:1789760775.200668:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760775.200693:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565428224:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760775.214263:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984484 gen 3 00000008:00010000:0.0:1789760775.214313:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.214320:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.214331:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.214355:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.214383:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565428224:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760775.214926:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17176, 17176] 00000080:00200000:3.0:1789760775.214955:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.214962:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.214977:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70356992 i_blocks 137440 00000020:00200000:3.0:1789760775.214981:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.214994:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.215030:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70356992, count: 4096 00000080:00200000:3.0:1789760775.215037:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70356992, bytes: 4096 00000080:00200000:3.0:1789760775.215048:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.215056:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.215059:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70356992, 70361088) 0 0 00000080:00200000:3.0:1789760775.215063:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.215080:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17177, 17177] 00020000:00200000:3.0:1789760775.215089:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70356992 chunk: [70356992, 70361088) 70361088, 4096 00020000:00200000:3.0:1789760775.215094:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.215115:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17177, 17177] 00000008:00010000:3.0:1789760775.215133:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.215139:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.215151:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70356992, 70361088) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.215166:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70356992=0x4319000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.216636:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.216661:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70361088 00000020:00010000:3.0:1789760775.216675:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.216680:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.216683:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.216687:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.216689:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.216701:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.216709:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.216790:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760775.217042:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 70356992<>70361088 00000080:00200000:0.0:1789760775.230617:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17177, 17177] 00000080:00200000:0.0:1789760775.230636:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.230683:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.230702:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70361088 i_blocks 137448 00000020:00200000:0.0:1789760775.230705:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.230716:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.230920:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70361088, count: 4096 00000080:00200000:0.0:1789760775.230930:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70361088, bytes: 4096 00000080:00200000:0.0:1789760775.230943:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.230951:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.230954:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70361088, 70365184) 0 0 00000080:00200000:0.0:1789760775.230959:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.230975:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17178, 17178] 00020000:00200000:0.0:1789760775.230985:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70361088 chunk: [70361088, 70365184) 70365184, 4096 00020000:00200000:0.0:1789760775.230990:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.231012:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17178, 17178] 00000008:00010000:0.0:1789760775.231030:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.231035:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.231047:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70361088, 70365184) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.231060:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70361088=0x431a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.231122:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.231125:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70365184 00000020:00010000:0.0:1789760775.231133:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.231136:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.231139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.231143:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.231146:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.231153:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.231160:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.231179:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.231303:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70361088<>70365184 00000100:00100000:1.0:1789760775.231448:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565428480:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760775.240308:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984486 gen 3 00000008:00010000:0.0:1789760775.240489:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.240497:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.240509:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.240616:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.240749:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565428480:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.241384:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565428608:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.251523:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17179, 17179] 00000080:00200000:0.0:1789760775.251550:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.251557:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.251573:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70369280 i_blocks 137464 00000020:00200000:0.0:1789760775.251577:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.251588:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.251622:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70369280, count: 4096 00000080:00200000:0.0:1789760775.251629:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70369280, bytes: 4096 00000080:00200000:0.0:1789760775.251641:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.251689:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.251692:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70369280, 70373376) 0 0 00000080:00200000:0.0:1789760775.251696:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.251713:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17180, 17180] 00020000:00200000:0.0:1789760775.251722:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70369280 chunk: [70369280, 70373376) 70373376, 4096 00020000:00200000:0.0:1789760775.251726:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.251748:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17180, 17180] 00000008:00010000:0.0:1789760775.251765:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.251771:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.251782:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70369280, 70373376) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.251796:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70369280=0x431c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.252581:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.252590:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70373376 00000020:00010000:0.0:1789760775.252602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.252607:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.252610:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.252614:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.252617:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.252628:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.252636:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.252693:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.252826:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70369280<>70373376 00000100:00100000:1.0:1789760775.253060:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565428736:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760775.267503:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984488 gen 3 00000008:00010000:0.0:1789760775.267566:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.267575:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.267586:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.267723:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.267755:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565428736:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760775.295558:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17181, 17181] 00000080:00200000:3.0:1789760775.295575:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.295581:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.295596:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70377472 i_blocks 137480 00000020:00200000:3.0:1789760775.295601:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.295756:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.295793:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70377472, count: 4096 00000080:00200000:3.0:1789760775.295801:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70377472, bytes: 4096 00000080:00200000:3.0:1789760775.295817:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.295827:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.295829:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70377472, 70381568) 0 0 00000080:00200000:3.0:1789760775.295833:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.295850:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17182, 17182] 00020000:00200000:3.0:1789760775.295858:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70377472 chunk: [70377472, 70381568) 70381568, 4096 00020000:00200000:3.0:1789760775.295862:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.295881:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17182, 17182] 00000008:00010000:3.0:1789760775.295898:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.295903:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.295915:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70377472, 70381568) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.295927:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70377472=0x431e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.295985:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.295988:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70381568 00000020:00010000:3.0:1789760775.295997:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.296000:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.296003:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.296006:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.296024:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.296034:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.296041:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.296058:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760775.296173:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 70377472<>70381568 00000080:00200000:3.0:1789760775.305770:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17182, 17182] 00000080:00200000:3.0:1789760775.305788:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.305795:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.305810:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70381568 i_blocks 137488 00000020:00200000:3.0:1789760775.305815:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.305826:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.305864:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70381568, count: 4096 00000080:00200000:3.0:1789760775.305872:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70381568, bytes: 4096 00000080:00200000:3.0:1789760775.305896:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.305906:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.305909:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70381568, 70385664) 0 0 00000080:00200000:3.0:1789760775.305913:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.305933:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17183, 17183] 00020000:00200000:3.0:1789760775.305943:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70381568 chunk: [70381568, 70385664) 70385664, 4096 00020000:00200000:3.0:1789760775.305948:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.305969:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17183, 17183] 00000008:00010000:3.0:1789760775.305988:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.305994:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.306005:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70381568, 70385664) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.306019:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70381568=0x431f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.306182:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.306185:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70385664 00000020:00010000:3.0:1789760775.306195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.306199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.306202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.306205:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.306208:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.306217:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.306224:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.306241:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760775.306611:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 70381568<>70385664 00000080:00200000:3.0:1789760775.321216:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17183, 17183] 00000080:00200000:3.0:1789760775.321233:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.321239:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.321267:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70385664 i_blocks 137496 00000020:00200000:3.0:1789760775.321271:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.321282:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.321314:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70385664, count: 4096 00000080:00200000:3.0:1789760775.321322:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70385664, bytes: 4096 00000080:00200000:3.0:1789760775.321333:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.321342:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.321345:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70385664, 70389760) 0 0 00000080:00200000:3.0:1789760775.321349:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.321366:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17184, 17184] 00020000:00200000:3.0:1789760775.321375:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70385664 chunk: [70385664, 70389760) 70389760, 4096 00020000:00200000:3.0:1789760775.321380:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.321398:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17184, 17184] 00000008:00010000:3.0:1789760775.321417:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.321423:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.321435:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70385664, 70389760) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.321449:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70385664=0x4320000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.321526:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.321529:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70389760 00000020:00010000:3.0:1789760775.321548:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.321552:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.321556:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.321560:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.321563:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.321571:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.321578:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.321595:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760775.321744:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 70385664<>70389760 00000080:00200000:3.0:1789760775.334844:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17184, 17184] 00000080:00200000:3.0:1789760775.334861:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.334868:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.334883:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70389760 i_blocks 137504 00000020:00200000:3.0:1789760775.334886:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.334896:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.334929:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70389760, count: 4096 00000080:00200000:3.0:1789760775.334937:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70389760, bytes: 4096 00000080:00200000:3.0:1789760775.334950:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.334958:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.334961:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70389760, 70393856) 0 0 00000080:00200000:3.0:1789760775.334965:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.334982:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17185, 17185] 00020000:00200000:3.0:1789760775.334991:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70389760 chunk: [70389760, 70393856) 70393856, 4096 00020000:00200000:3.0:1789760775.334996:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.335016:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17185, 17185] 00000008:00010000:3.0:1789760775.335033:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.335039:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.335051:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70389760, 70393856) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.335065:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70389760=0x4321000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.335119:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.335122:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70393856 00000020:00010000:3.0:1789760775.335131:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.335135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.335138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.335142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.335145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.335153:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.335159:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.335178:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760775.335297:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66d80 - object 0x240000400:5 offset 70389760<>70393856 00000080:00200000:3.0:1789760775.344420:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17185, 17185] 00000080:00200000:3.0:1789760775.344437:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.344444:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.344459:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70393856 i_blocks 137512 00000020:00200000:3.0:1789760775.344463:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.344482:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.344584:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70393856, count: 4096 00000080:00200000:3.0:1789760775.344592:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70393856, bytes: 4096 00000080:00200000:3.0:1789760775.344603:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.344612:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.344614:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70393856, 70397952) 0 0 00000080:00200000:3.0:1789760775.344619:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.344637:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17186, 17186] 00020000:00200000:3.0:1789760775.344690:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70393856 chunk: [70393856, 70397952) 70397952, 4096 00020000:00200000:3.0:1789760775.344695:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.344716:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17186, 17186] 00000008:00010000:3.0:1789760775.344734:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.344742:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.344755:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70393856, 70397952) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.344769:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70393856=0x4322000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.344837:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.344840:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70397952 00000020:00010000:3.0:1789760775.344849:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.344852:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.344856:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.344860:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.344863:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.344872:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.344878:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.345022:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760775.345150:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 70393856<>70397952 00000080:00200000:3.0:1789760775.353849:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17186, 17186] 00000080:00200000:3.0:1789760775.353865:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.353871:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.353887:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70397952 i_blocks 137520 00000020:00200000:3.0:1789760775.353890:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.353900:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.353931:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70397952, count: 4096 00000080:00200000:3.0:1789760775.353937:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70397952, bytes: 4096 00000080:00200000:3.0:1789760775.353947:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.353954:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.353957:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70397952, 70402048) 0 0 00000080:00200000:3.0:1789760775.353961:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.353977:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17187, 17187] 00020000:00200000:3.0:1789760775.353986:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70397952 chunk: [70397952, 70402048) 70402048, 4096 00020000:00200000:3.0:1789760775.353991:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.354009:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17187, 17187] 00000008:00010000:3.0:1789760775.354026:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.354033:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.354045:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70397952, 70402048) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.354058:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70397952=0x4323000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.354114:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.354117:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70402048 00000020:00010000:3.0:1789760775.354125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.354129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.354133:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.354137:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.354141:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.354148:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.354155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.354171:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760775.354289:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 70397952<>70402048 00000080:00200000:3.0:1789760775.363963:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17187, 17187] 00000080:00200000:3.0:1789760775.363979:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.363986:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.364001:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70402048 i_blocks 137528 00000020:00200000:3.0:1789760775.364005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.364015:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.364048:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70402048, count: 4096 00000080:00200000:3.0:1789760775.364054:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70402048, bytes: 4096 00000080:00200000:3.0:1789760775.364064:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.364073:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.364075:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70402048, 70406144) 0 0 00000080:00200000:3.0:1789760775.364080:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.364097:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17188, 17188] 00020000:00200000:3.0:1789760775.364106:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70402048 chunk: [70402048, 70406144) 70406144, 4096 00020000:00200000:3.0:1789760775.364110:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.364130:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17188, 17188] 00000008:00010000:3.0:1789760775.364147:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.364153:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.364165:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70402048, 70406144) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.364178:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70402048=0x4324000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.364232:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.364235:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70406144 00000020:00010000:3.0:1789760775.364243:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.364247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.364250:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.364254:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.364258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.364266:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.364273:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.364289:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760775.364409:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 70402048<>70406144 00000080:00200000:3.0:1789760775.373688:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17188, 17188] 00000080:00200000:3.0:1789760775.373708:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.373715:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.373732:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70406144 i_blocks 137536 00000020:00200000:3.0:1789760775.373736:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.373747:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.373782:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70406144, count: 4096 00000080:00200000:3.0:1789760775.373790:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70406144, bytes: 4096 00000080:00200000:3.0:1789760775.373800:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.373809:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.373811:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70406144, 70410240) 0 0 00000080:00200000:3.0:1789760775.373816:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.373833:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17189, 17189] 00020000:00200000:3.0:1789760775.373842:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70406144 chunk: [70406144, 70410240) 70410240, 4096 00020000:00200000:3.0:1789760775.373846:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.373866:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17189, 17189] 00000008:00010000:3.0:1789760775.373895:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.373902:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.373914:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70406144, 70410240) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.373927:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70406144=0x4325000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.373985:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.373988:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70410240 00000020:00010000:3.0:1789760775.373996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.374000:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.374004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.374008:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.374011:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.374020:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.374026:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.374043:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760775.374163:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 70406144<>70410240 00000080:00200000:3.0:1789760775.392320:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17189, 17189] 00000080:00200000:3.0:1789760775.392339:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.392346:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.392365:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70410240 i_blocks 137544 00000020:00200000:3.0:1789760775.392369:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.392380:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.392416:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70410240, count: 4096 00000080:00200000:3.0:1789760775.392423:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70410240, bytes: 4096 00000080:00200000:3.0:1789760775.392437:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.392446:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.392449:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70410240, 70414336) 0 0 00000080:00200000:3.0:1789760775.392454:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.392604:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17190, 17190] 00020000:00200000:3.0:1789760775.392615:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70410240 chunk: [70410240, 70414336) 70414336, 4096 00020000:00200000:3.0:1789760775.392621:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.392959:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17190, 17190] 00000008:00010000:3.0:1789760775.392982:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.392989:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.393001:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70410240, 70414336) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.393083:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70410240=0x4326000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.393148:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.393151:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70414336 00000020:00010000:3.0:1789760775.393160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.393164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.393167:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.393171:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.393174:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.393183:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.393190:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.393212:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760775.393338:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 70410240<>70414336 00000080:00200000:3.0:1789760775.405816:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17190, 17190] 00000080:00200000:3.0:1789760775.405833:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.405839:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.405855:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70414336 i_blocks 137552 00000020:00200000:3.0:1789760775.405859:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.405870:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.406094:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70414336, count: 4096 00000080:00200000:3.0:1789760775.406103:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70414336, bytes: 4096 00000080:00200000:3.0:1789760775.406115:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.406124:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.406126:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70414336, 70418432) 0 0 00000080:00200000:3.0:1789760775.406131:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.406148:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17191, 17191] 00020000:00200000:3.0:1789760775.406157:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70414336 chunk: [70414336, 70418432) 70418432, 4096 00020000:00200000:3.0:1789760775.406162:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.406182:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17191, 17191] 00000008:00010000:3.0:1789760775.406200:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.406207:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.406220:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70414336, 70418432) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.406233:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70414336=0x4327000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.406294:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.406297:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70418432 00000020:00010000:3.0:1789760775.406305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.406309:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.406312:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.406316:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.406319:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.406328:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.406335:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.406357:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760775.406705:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 70414336<>70418432 00000080:00200000:3.0:1789760775.414422:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17191, 17191] 00000080:00200000:3.0:1789760775.414439:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.414445:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.414461:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70418432 i_blocks 137560 00000020:00200000:3.0:1789760775.414465:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.414487:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.414521:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70418432, count: 4096 00000080:00200000:3.0:1789760775.414527:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70418432, bytes: 4096 00000080:00200000:3.0:1789760775.414544:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.414553:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.414556:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70418432, 70422528) 0 0 00000080:00200000:3.0:1789760775.414560:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.414893:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17192, 17192] 00020000:00200000:3.0:1789760775.414902:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70418432 chunk: [70418432, 70422528) 70422528, 4096 00020000:00200000:3.0:1789760775.414907:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.414926:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17192, 17192] 00000008:00010000:3.0:1789760775.414943:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.414950:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.414961:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70418432, 70422528) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.414973:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70418432=0x4328000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.415132:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.415136:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70422528 00000020:00010000:3.0:1789760775.415144:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.415148:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.415152:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.415155:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.415158:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.415166:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.415173:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.415192:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760775.415304:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 70418432<>70422528 00000080:00200000:3.0:1789760775.425066:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17192, 17192] 00000080:00200000:3.0:1789760775.425083:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.425089:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.425171:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70422528 i_blocks 137568 00000020:00200000:3.0:1789760775.425176:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.425187:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.425220:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70422528, count: 4096 00000080:00200000:3.0:1789760775.425227:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70422528, bytes: 4096 00000080:00200000:3.0:1789760775.425237:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.425246:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.425248:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70422528, 70426624) 0 0 00000080:00200000:3.0:1789760775.425253:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.425269:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17193, 17193] 00020000:00200000:3.0:1789760775.425277:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70422528 chunk: [70422528, 70426624) 70426624, 4096 00020000:00200000:3.0:1789760775.425282:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.425300:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17193, 17193] 00000008:00010000:3.0:1789760775.425317:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.425323:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.425335:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70422528, 70426624) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.425348:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70422528=0x4329000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.425404:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.425407:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70426624 00000020:00010000:3.0:1789760775.425415:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.425419:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.425422:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.425425:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.425428:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.425436:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.425442:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.425459:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760775.425588:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 70422528<>70426624 00000080:00200000:3.0:1789760775.440799:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17193, 17193] 00000080:00200000:3.0:1789760775.440821:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.440828:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.440846:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70426624 i_blocks 137576 00000020:00200000:3.0:1789760775.440851:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.440863:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.440897:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70426624, count: 4096 00000080:00200000:3.0:1789760775.440905:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70426624, bytes: 4096 00000080:00200000:3.0:1789760775.440918:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.440927:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.440930:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70426624, 70430720) 0 0 00000080:00200000:3.0:1789760775.440935:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.440954:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17194, 17194] 00020000:00200000:3.0:1789760775.440963:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70426624 chunk: [70426624, 70430720) 70430720, 4096 00020000:00200000:3.0:1789760775.440968:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.440989:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17194, 17194] 00000008:00010000:3.0:1789760775.441009:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.441017:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.441028:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70426624, 70430720) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.441042:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70426624=0x432a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.441188:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.441192:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70430720 00000020:00010000:3.0:1789760775.441201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.441205:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.441208:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.441211:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.441214:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.441224:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.441231:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.441259:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760775.441388:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 70426624<>70430720 00000080:00200000:3.0:1789760775.455787:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17194, 17194] 00000080:00200000:3.0:1789760775.455805:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.455811:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.455827:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70430720 i_blocks 137584 00000020:00200000:3.0:1789760775.455831:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.455842:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.455876:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70430720, count: 4096 00000080:00200000:3.0:1789760775.455883:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70430720, bytes: 4096 00000080:00200000:3.0:1789760775.455893:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.455902:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.455905:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70430720, 70434816) 0 0 00000080:00200000:3.0:1789760775.455909:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.455927:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17195, 17195] 00020000:00200000:3.0:1789760775.455937:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70430720 chunk: [70430720, 70434816) 70434816, 4096 00020000:00200000:3.0:1789760775.455941:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.455963:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17195, 17195] 00000008:00010000:3.0:1789760775.455981:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.455988:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.456000:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70430720, 70434816) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.456037:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70430720=0x432b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.456112:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.456118:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70434816 00000020:00010000:3.0:1789760775.456127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.456131:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.456134:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.456137:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.456140:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.456148:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.456155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.456172:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760775.456280:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 70430720<>70434816 00000080:00200000:3.0:1789760775.467163:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17195, 17195] 00000080:00200000:3.0:1789760775.467182:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.467188:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.467203:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70434816 i_blocks 137592 00000020:00200000:3.0:1789760775.467208:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.467218:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.467251:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70434816, count: 4096 00000080:00200000:3.0:1789760775.467259:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70434816, bytes: 4096 00000080:00200000:3.0:1789760775.467272:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.467281:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.467283:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70434816, 70438912) 0 0 00000080:00200000:3.0:1789760775.467288:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.467305:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17196, 17196] 00020000:00200000:3.0:1789760775.467314:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70434816 chunk: [70434816, 70438912) 70438912, 4096 00020000:00200000:3.0:1789760775.467318:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.467338:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17196, 17196] 00000008:00010000:3.0:1789760775.467356:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.467363:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.467376:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70434816, 70438912) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.467402:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70434816=0x432c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.467463:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.467477:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70438912 00000020:00010000:3.0:1789760775.467486:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.467490:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.467494:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.467497:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.467501:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.467510:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.467517:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.467548:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760775.467725:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 70434816<>70438912 00000080:00200000:3.0:1789760775.478473:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17196, 17196] 00000080:00200000:3.0:1789760775.478550:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.478558:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760775.479389:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565430912:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.496144:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17197, 17197] 00000080:00200000:0.0:1789760775.496163:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.496169:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.496185:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70443008 i_blocks 137608 00000020:00200000:0.0:1789760775.496189:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.496199:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.496236:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70443008, count: 4096 00000080:00200000:0.0:1789760775.496244:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70443008, bytes: 4096 00000080:00200000:0.0:1789760775.496254:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.496263:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.496265:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70443008, 70447104) 0 0 00000080:00200000:0.0:1789760775.496270:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.496286:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17198, 17198] 00020000:00200000:0.0:1789760775.496295:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70443008 chunk: [70443008, 70447104) 70447104, 4096 00020000:00200000:0.0:1789760775.496300:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.496319:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17198, 17198] 00000008:00010000:0.0:1789760775.496338:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.496345:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.496357:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70443008, 70447104) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.496371:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70443008=0x432e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.496432:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.496436:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70447104 00000020:00010000:0.0:1789760775.496444:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.496447:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.496450:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.496453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.496456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.496463:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.496481:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.496498:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.496749:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70443008<>70447104 00000100:00100000:0.0:1789760775.496793:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760775.496810:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565431040:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760775.515029:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984506 gen 3 00000008:00010000:0.0:1789760775.515081:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.515089:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.515100:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.515129:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.515159:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565431040:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.527546:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17199, 17199] 00000080:00200000:0.0:1789760775.527564:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.527571:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.527586:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70451200 i_blocks 137624 00000020:00200000:0.0:1789760775.527852:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.527864:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.527899:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70451200, count: 4096 00000080:00200000:0.0:1789760775.527906:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70451200, bytes: 4096 00000080:00200000:0.0:1789760775.527917:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.527925:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.527928:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70451200, 70455296) 0 0 00000080:00200000:0.0:1789760775.527933:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.527950:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17200, 17200] 00020000:00200000:0.0:1789760775.527959:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70451200 chunk: [70451200, 70455296) 70455296, 4096 00020000:00200000:0.0:1789760775.527964:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.527983:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17200, 17200] 00000008:00010000:0.0:1789760775.528002:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.528007:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.528019:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70451200, 70455296) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.528033:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70451200=0x4330000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.528093:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.528096:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70455296 00000020:00010000:0.0:1789760775.528104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.528108:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.528111:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.528115:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.528118:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.528125:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.528133:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.528153:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.528338:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70451200<>70455296 00000100:00100000:1.0:1789760775.528420:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565431296:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760775.537350:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984508 gen 3 00000008:00010000:0.0:1789760775.537402:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.537412:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.537424:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.537452:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.537487:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565431296:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760775.546613:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17201, 17201] 00000080:00200000:3.0:1789760775.546630:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.546636:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.546694:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70459392 i_blocks 137640 00000020:00200000:3.0:1789760775.546699:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.546709:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.546773:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70459392, count: 4096 00000080:00200000:3.0:1789760775.546781:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70459392, bytes: 4096 00000080:00200000:3.0:1789760775.546791:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.546800:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.546802:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70459392, 70463488) 0 0 00000080:00200000:3.0:1789760775.546808:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.546826:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17202, 17202] 00020000:00200000:3.0:1789760775.546835:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70459392 chunk: [70459392, 70463488) 70463488, 4096 00020000:00200000:3.0:1789760775.546840:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.546860:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17202, 17202] 00000008:00010000:3.0:1789760775.546878:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.546885:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.546896:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70459392, 70463488) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.546911:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70459392=0x4332000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.546972:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.546975:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70463488 00000020:00010000:3.0:1789760775.546983:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.546986:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.546989:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.546993:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.546996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.547004:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.547010:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.547027:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760775.547157:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 70459392<>70463488 00000080:00200000:3.0:1789760775.558316:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17202, 17202] 00000080:00200000:3.0:1789760775.558332:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.558338:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.558354:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70463488 i_blocks 137648 00000020:00200000:3.0:1789760775.558358:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.558368:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.558402:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70463488, count: 4096 00000080:00200000:3.0:1789760775.558409:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70463488, bytes: 4096 00000080:00200000:3.0:1789760775.558419:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.558428:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.558430:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70463488, 70467584) 0 0 00000080:00200000:3.0:1789760775.558436:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.558453:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17203, 17203] 00020000:00200000:3.0:1789760775.558462:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70463488 chunk: [70463488, 70467584) 70467584, 4096 00020000:00200000:3.0:1789760775.558467:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.558487:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17203, 17203] 00000008:00010000:3.0:1789760775.558505:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.558511:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.558522:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70463488, 70467584) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.558544:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70463488=0x4333000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.558602:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.558605:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70467584 00000020:00010000:3.0:1789760775.558614:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.558618:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.558621:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.558706:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.558711:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.558721:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.558729:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.558745:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760775.558934:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 70463488<>70467584 00000080:00200000:3.0:1789760775.572141:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17203, 17203] 00000080:00200000:3.0:1789760775.572160:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.572166:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760775.572185:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70467584 i_blocks 137656 00000020:00200000:3.0:1789760775.572189:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760775.572199:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760775.572461:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70467584, count: 4096 00000080:00200000:3.0:1789760775.572544:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70467584, bytes: 4096 00000080:00200000:3.0:1789760775.572559:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760775.572859:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760775.572862:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70467584, 70471680) 0 0 00000080:00200000:3.0:1789760775.572869:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760775.573311:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17204, 17204] 00020000:00200000:3.0:1789760775.573322:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70467584 chunk: [70467584, 70471680) 70471680, 4096 00020000:00200000:3.0:1789760775.573326:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760775.573350:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17204, 17204] 00000008:00010000:3.0:1789760775.573448:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760775.573455:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760775.573540:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70467584, 70471680) append 0 append lockpos 0 00000080:00200000:3.0:1789760775.573557:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70467584=0x4334000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760775.573723:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760775.573727:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70471680 00000020:00010000:3.0:1789760775.573737:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760775.573741:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.573744:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760775.573747:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760775.573750:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760775.573760:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760775.573767:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760775.573794:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760775.573939:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 70467584<>70471680 00000080:00200000:3.0:1789760775.592470:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17204, 17204] 00000080:00200000:3.0:1789760775.592499:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.592505:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760775.595177:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565431936:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.611682:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17205, 17205] 00000080:00200000:0.0:1789760775.612185:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.612193:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.612218:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70475776 i_blocks 137672 00000020:00200000:0.0:1789760775.612223:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.612234:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.612268:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70475776, count: 4096 00000080:00200000:0.0:1789760775.612277:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70475776, bytes: 4096 00000080:00200000:0.0:1789760775.612288:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.612297:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.612299:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70475776, 70479872) 0 0 00000080:00200000:0.0:1789760775.612303:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.612321:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17206, 17206] 00020000:00200000:0.0:1789760775.612330:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70475776 chunk: [70475776, 70479872) 70479872, 4096 00020000:00200000:0.0:1789760775.612335:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.612355:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17206, 17206] 00000008:00010000:0.0:1789760775.612372:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.612377:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.612388:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70475776, 70479872) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.612400:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70475776=0x4336000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.612456:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.612459:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70479872 00000020:00010000:0.0:1789760775.612467:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.612470:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.612473:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.612476:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.612479:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.612486:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.612494:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.612515:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.612887:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70475776<>70479872 00000100:00100000:1.0:1789760775.612974:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565432064:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760775.629797:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984514 gen 3 00000008:00010000:0.0:1789760775.629982:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.629991:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.630003:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.630020:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.630051:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565432064:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760775.650697:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17207, 17207] 00000080:00200000:3.0:1789760775.650719:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760775.650726:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.650820:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70483968 i_blocks 137688 00000020:00200000:0.0:1789760775.650826:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.650839:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.650874:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70483968, count: 4096 00000080:00200000:0.0:1789760775.650882:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70483968, bytes: 4096 00000080:00200000:0.0:1789760775.650899:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.650908:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.650910:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70483968, 70488064) 0 0 00000080:00200000:0.0:1789760775.650915:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.650931:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17208, 17208] 00020000:00200000:0.0:1789760775.650941:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70483968 chunk: [70483968, 70488064) 70488064, 4096 00020000:00200000:0.0:1789760775.650947:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.650969:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17208, 17208] 00000008:00010000:0.0:1789760775.650997:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.651003:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.651014:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70483968, 70488064) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.651028:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70483968=0x4338000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.651092:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.651096:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70488064 00000020:00010000:0.0:1789760775.651104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.651107:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.651111:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.651114:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.651117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.651125:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.651132:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.651152:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.651273:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70483968<>70488064 00000100:00100000:0.0:1789760775.651340:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565432320:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760775.660090:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984516 gen 3 00000008:00010000:0.0:1789760775.660141:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.660150:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.660161:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.660191:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.660219:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565432320:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.674728:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17209, 17209] 00000080:00200000:0.0:1789760775.674745:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.674750:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.674767:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70492160 i_blocks 137704 00000020:00200000:0.0:1789760775.674770:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.674781:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.674814:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70492160, count: 4096 00000080:00200000:0.0:1789760775.674821:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70492160, bytes: 4096 00000080:00200000:0.0:1789760775.674831:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.674841:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.674843:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70492160, 70496256) 0 0 00000080:00200000:0.0:1789760775.674847:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.674864:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17210, 17210] 00020000:00200000:0.0:1789760775.674873:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70492160 chunk: [70492160, 70496256) 70496256, 4096 00020000:00200000:0.0:1789760775.674877:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.674897:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17210, 17210] 00000008:00010000:0.0:1789760775.674917:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.674924:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.674934:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70492160, 70496256) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.674946:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70492160=0x433a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.674998:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.675000:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70496256 00000020:00010000:0.0:1789760775.675009:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.675012:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.675015:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.675018:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.675021:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.675028:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.675035:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.675052:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.675173:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70492160<>70496256 00000100:00100000:0.0:1789760775.675221:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760775.675237:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565432576:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.687952:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984518 gen 3 00000008:00010000:0.0:1789760775.688279:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:0.0:1789760775.688292:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760775.688302:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760775.688373:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:0.0:1789760775.688406:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565432576:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.700208:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17211, 17211] 00000080:00200000:0.0:1789760775.700226:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.700233:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.700248:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70500352 i_blocks 137720 00000020:00200000:0.0:1789760775.700252:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.700263:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.700300:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70500352, count: 4096 00000080:00200000:0.0:1789760775.700309:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70500352, bytes: 4096 00000080:00200000:0.0:1789760775.700322:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.700331:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.700333:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70500352, 70504448) 0 0 00000080:00200000:0.0:1789760775.700338:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.700354:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17212, 17212] 00020000:00200000:0.0:1789760775.700363:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70500352 chunk: [70500352, 70504448) 70504448, 4096 00020000:00200000:0.0:1789760775.700367:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.700386:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17212, 17212] 00000008:00010000:0.0:1789760775.700521:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.700537:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.700551:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70500352, 70504448) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.700566:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70500352=0x433c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.700633:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.700636:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70504448 00000020:00010000:0.0:1789760775.700804:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.700808:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.700812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.700815:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.700818:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.700828:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.700835:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.700860:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760775.701051:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70500352<>70504448 00000100:00100000:0.0:1789760775.702196:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565432832:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760775.707036:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760775.715039:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984520 gen 3 00000008:00010000:1.0:1789760775.715107:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760775.715116:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.715131:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.715152:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:1.0:1789760775.715181:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565432832:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.715330:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17212, 17212] 00000080:00200000:0.0:1789760775.715346:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.715352:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.715366:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70504448 i_blocks 137728 00000020:00200000:0.0:1789760775.715370:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.715382:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.715414:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70504448, count: 4096 00000080:00200000:0.0:1789760775.715420:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70504448, bytes: 4096 00000080:00200000:0.0:1789760775.715431:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.715439:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.715441:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70504448, 70508544) 0 0 00000080:00200000:0.0:1789760775.715446:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.715463:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17213, 17213] 00020000:00200000:0.0:1789760775.715471:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70504448 chunk: [70504448, 70508544) 70508544, 4096 00020000:00200000:0.0:1789760775.715476:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.715494:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17213, 17213] 00000008:00010000:0.0:1789760775.715520:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.715526:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.715546:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70504448, 70508544) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.715561:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70504448=0x433d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.715622:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.715625:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70508544 00000020:00010000:0.0:1789760775.715838:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.715843:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.715847:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.715851:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.715854:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.715862:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.715869:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.715889:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760775.716026:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42680 - object 0x240000400:5 offset 70504448<>70508544 00000100:00100000:1.0:1789760775.716849:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565432960:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.727639:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984521 gen 3 00000008:00010000:1.0:1789760775.727738:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760775.727748:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.727759:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.727771:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760775.727800:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565432960:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.727924:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17213, 17213] 00000080:00200000:0.0:1789760775.727940:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.727946:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.727961:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70508544 i_blocks 137736 00000020:00200000:0.0:1789760775.727965:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.727977:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.728011:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70508544, count: 4096 00000080:00200000:0.0:1789760775.728018:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70508544, bytes: 4096 00000080:00200000:0.0:1789760775.728029:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.728038:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.728040:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70508544, 70512640) 0 0 00000080:00200000:0.0:1789760775.728045:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.728065:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17214, 17214] 00020000:00200000:0.0:1789760775.728075:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70508544 chunk: [70508544, 70512640) 70512640, 4096 00020000:00200000:0.0:1789760775.728079:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.728099:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17214, 17214] 00000008:00010000:0.0:1789760775.728118:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.728124:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.728136:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70508544, 70512640) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.728150:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70508544=0x433e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.728206:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.728208:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70512640 00000020:00010000:0.0:1789760775.728216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.728220:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.728223:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.728226:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.728229:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.728236:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.728243:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.728261:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760775.728391:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 70508544<>70512640 00000100:00100000:1.0:1789760775.728483:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760775.728499:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565433088:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.738997:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984522 gen 3 00000008:00010000:1.0:1789760775.739044:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760775.739052:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.739062:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.739123:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760775.739153:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565433088:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.739465:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17214, 17214] 00000080:00200000:0.0:1789760775.739481:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.739488:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.739503:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70512640 i_blocks 137744 00000020:00200000:0.0:1789760775.739506:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.739519:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.739567:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70512640, count: 4096 00000080:00200000:0.0:1789760775.739575:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70512640, bytes: 4096 00000080:00200000:0.0:1789760775.739585:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.739594:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.739596:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70512640, 70516736) 0 0 00000080:00200000:0.0:1789760775.739601:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.739770:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17215, 17215] 00020000:00200000:0.0:1789760775.739781:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70512640 chunk: [70512640, 70516736) 70516736, 4096 00020000:00200000:0.0:1789760775.739787:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.739812:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17215, 17215] 00000008:00010000:0.0:1789760775.739830:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.739837:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.739850:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70512640, 70516736) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.739866:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70512640=0x433f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.740027:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.740030:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70516736 00000020:00010000:0.0:1789760775.740039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.740043:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.740046:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.740050:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.740053:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.740062:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.740070:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.740093:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760775.740275:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 70512640<>70516736 00000100:00100000:1.0:1789760775.740799:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565433216:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.749937:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984523 gen 3 00000008:00010000:1.0:1789760775.749991:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760775.749999:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.750010:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.750033:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:1.0:1789760775.750074:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565433216:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.750274:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17215, 17215] 00000080:00200000:0.0:1789760775.750347:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.750416:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.750433:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70516736 i_blocks 137752 00000020:00200000:0.0:1789760775.750438:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.750450:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.750482:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70516736, count: 4096 00000080:00200000:0.0:1789760775.750489:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70516736, bytes: 4096 00000080:00200000:0.0:1789760775.750497:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.750504:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.750571:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70516736, 70520832) 0 0 00000080:00200000:0.0:1789760775.750577:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.750595:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17216, 17216] 00020000:00200000:0.0:1789760775.750604:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70516736 chunk: [70516736, 70520832) 70520832, 4096 00020000:00200000:0.0:1789760775.750609:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.750629:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17216, 17216] 00000008:00010000:0.0:1789760775.750687:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.750694:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.750706:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70516736, 70520832) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.750720:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70516736=0x4340000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.750778:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.750781:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70520832 00000020:00010000:0.0:1789760775.750790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.750794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.750796:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.750800:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.750803:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.750809:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.750816:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.750835:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d300 00000008:00100000:0.0:1789760775.750958:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 70516736<>70520832 00000100:00100000:1.0:1789760775.751006:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760775.751024:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565433344:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.762141:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984524 gen 3 00000008:00010000:1.0:1789760775.762249:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760775.762257:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.762268:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.762332:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760775.762363:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565433344:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.762444:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17216, 17216] 00000080:00200000:0.0:1789760775.762460:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.762536:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.762553:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70520832 i_blocks 137760 00000020:00200000:0.0:1789760775.762557:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.762630:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.762688:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70520832, count: 4096 00000080:00200000:0.0:1789760775.762696:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70520832, bytes: 4096 00000080:00200000:0.0:1789760775.762706:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.762715:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.762717:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70520832, 70524928) 0 0 00000080:00200000:0.0:1789760775.762722:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.762739:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17217, 17217] 00020000:00200000:0.0:1789760775.762748:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70520832 chunk: [70520832, 70524928) 70524928, 4096 00020000:00200000:0.0:1789760775.762753:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.762772:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17217, 17217] 00000008:00010000:0.0:1789760775.762790:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.762796:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.762873:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70520832, 70524928) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.762888:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70520832=0x4341000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.762947:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.762950:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70524928 00000020:00010000:0.0:1789760775.762959:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.762962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.762965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.762969:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.762972:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.762982:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.762990:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.763012:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760775.763269:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 70520832<>70524928 00000100:00100000:1.0:1789760775.763340:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565433472:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.772364:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984525 gen 3 00000008:00010000:1.0:1789760775.772418:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760775.772427:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.772438:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.772459:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:1.0:1789760775.772489:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565433472:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.772560:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17217, 17217] 00000080:00200000:0.0:1789760775.772577:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.772583:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.772599:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70524928 i_blocks 137768 00000020:00200000:0.0:1789760775.772603:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.772615:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.772704:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70524928, count: 4096 00000080:00200000:0.0:1789760775.772713:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70524928, bytes: 4096 00000080:00200000:0.0:1789760775.772723:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.772731:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.772734:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70524928, 70529024) 0 0 00000080:00200000:0.0:1789760775.772740:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.772759:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17218, 17218] 00020000:00200000:0.0:1789760775.772768:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70524928 chunk: [70524928, 70529024) 70529024, 4096 00020000:00200000:0.0:1789760775.772773:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.772792:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17218, 17218] 00000008:00010000:0.0:1789760775.772811:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.772822:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.772835:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70524928, 70529024) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.772849:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70524928=0x4342000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.772903:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.772907:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70529024 00000020:00010000:0.0:1789760775.772915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.772918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.772921:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.772925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.772928:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.772936:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.772943:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.772961:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760775.773090:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 70524928<>70529024 00000100:00100000:0.0:1789760775.773138:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565433600:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.784251:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984526 gen 3 00000008:00010000:1.0:1789760775.784297:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760775.784305:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.784317:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.784337:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760775.784367:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565433600:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.784473:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17218, 17218] 00000080:00200000:0.0:1789760775.784490:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.784496:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.784511:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70529024 i_blocks 137776 00000020:00200000:0.0:1789760775.784515:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.784525:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.784571:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70529024, count: 4096 00000080:00200000:0.0:1789760775.784579:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70529024, bytes: 4096 00000080:00200000:0.0:1789760775.784589:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.784597:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.784600:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70529024, 70533120) 0 0 00000080:00200000:0.0:1789760775.784604:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.784621:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17219, 17219] 00020000:00200000:0.0:1789760775.784630:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70529024 chunk: [70529024, 70533120) 70533120, 4096 00020000:00200000:0.0:1789760775.784635:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.784710:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17219, 17219] 00000008:00010000:0.0:1789760775.784727:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.784733:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.784746:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70529024, 70533120) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.784760:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70529024=0x4343000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.784816:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.784819:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70533120 00000020:00010000:0.0:1789760775.784828:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.784831:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.784834:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.784838:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.784841:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.784848:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.784855:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.784872:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760775.784997:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 70529024<>70533120 00000100:00100000:1.0:1789760775.785563:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760775.785585:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565433728:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.813192:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984527 gen 3 00000008:00010000:1.0:1789760775.813255:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760775.813264:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.813275:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.813296:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760775.813328:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565433728:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.813692:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17219, 17219] 00000080:00200000:0.0:1789760775.813711:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.813718:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.813735:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70533120 i_blocks 137784 00000020:00200000:0.0:1789760775.813739:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.813752:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.813793:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70533120, count: 4096 00000080:00200000:0.0:1789760775.813802:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70533120, bytes: 4096 00000080:00200000:0.0:1789760775.813814:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.813822:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.813825:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70533120, 70537216) 0 0 00000080:00200000:0.0:1789760775.813830:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.813847:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17220, 17220] 00020000:00200000:0.0:1789760775.813856:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70533120 chunk: [70533120, 70537216) 70537216, 4096 00020000:00200000:0.0:1789760775.813861:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.813893:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17220, 17220] 00000008:00010000:0.0:1789760775.813913:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.813919:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.813931:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70533120, 70537216) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.813947:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70533120=0x4344000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.814013:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.814016:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70537216 00000020:00010000:0.0:1789760775.814025:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.814028:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.814031:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.814035:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.814038:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.814048:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.814055:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.814077:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760775.814206:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 70533120<>70537216 00000100:00100000:1.0:1789760775.814709:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565433856:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.840824:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984528 gen 3 00000008:00010000:1.0:1789760775.840862:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760775.840870:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.840886:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.840969:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760775.840996:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565433856:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.841131:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17220, 17220] 00000080:00200000:0.0:1789760775.841144:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.841150:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.841255:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70537216 i_blocks 137792 00000020:00200000:0.0:1789760775.841261:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.841272:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.841298:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70537216, count: 4096 00000080:00200000:0.0:1789760775.841305:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70537216, bytes: 4096 00000080:00200000:0.0:1789760775.841324:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.841331:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.841333:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70537216, 70541312) 0 0 00000080:00200000:0.0:1789760775.841336:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.841348:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17221, 17221] 00020000:00200000:0.0:1789760775.841354:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70537216 chunk: [70537216, 70541312) 70541312, 4096 00020000:00200000:0.0:1789760775.841357:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.841371:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17221, 17221] 00000008:00010000:0.0:1789760775.841382:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.841386:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.841393:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70537216, 70541312) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.841402:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70537216=0x4345000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.841439:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.841441:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70541312 00000020:00010000:0.0:1789760775.841447:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.841449:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.841451:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.841453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.841455:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.841460:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.841465:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.841477:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dcc0 00000008:00100000:0.0:1789760775.841579:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 70537216<>70541312 00000100:00100000:1.0:1789760775.841636:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565433984:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.856717:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984529 gen 3 00000008:00010000:1.0:1789760775.856759:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760775.856767:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.856776:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.856796:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760775.856819:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565433984:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.856961:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17221, 17221] 00000080:00200000:0.0:1789760775.856973:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.856978:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.856990:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70541312 i_blocks 137800 00000020:00200000:0.0:1789760775.856993:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.857002:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.857027:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70541312, count: 4096 00000080:00200000:0.0:1789760775.857033:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70541312, bytes: 4096 00000080:00200000:0.0:1789760775.857040:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.857048:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.857050:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70541312, 70545408) 0 0 00000080:00200000:0.0:1789760775.857054:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.857066:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17222, 17222] 00020000:00200000:0.0:1789760775.857072:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70541312 chunk: [70541312, 70545408) 70545408, 4096 00020000:00200000:0.0:1789760775.857076:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.857090:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17222, 17222] 00000008:00010000:0.0:1789760775.857103:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.857108:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.857117:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70541312, 70545408) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.857128:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70541312=0x4346000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.857172:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.857175:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70545408 00000020:00010000:0.0:1789760775.857181:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.857185:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.857188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.857191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.857194:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.857201:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.857207:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.857223:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760775.857320:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 70541312<>70545408 00000100:00100000:0.0:1789760775.857355:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565434112:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.864831:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984530 gen 3 00000008:00010000:1.0:1789760775.864864:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760775.864872:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.864890:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.864942:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:1.0:1789760775.864963:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565434112:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.868119:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17222, 17222] 00000080:00200000:0.0:1789760775.868138:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.868144:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.868157:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70545408 i_blocks 137808 00000020:00200000:0.0:1789760775.868162:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.868175:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.868200:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70545408, count: 4096 00000080:00200000:0.0:1789760775.868207:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70545408, bytes: 4096 00000080:00200000:0.0:1789760775.868218:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.868226:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.868228:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70545408, 70549504) 0 0 00000080:00200000:0.0:1789760775.868232:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.868244:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17223, 17223] 00020000:00200000:0.0:1789760775.868250:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70545408 chunk: [70545408, 70549504) 70549504, 4096 00020000:00200000:0.0:1789760775.868254:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.868269:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17223, 17223] 00000008:00010000:0.0:1789760775.868282:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.868286:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.868295:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70545408, 70549504) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.868306:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70545408=0x4347000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.868358:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.868361:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70549504 00000020:00010000:0.0:1789760775.868367:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.868370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.868373:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.868376:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.868379:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.868386:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.868392:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.868417:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760775.868519:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 70545408<>70549504 00000100:00100000:1.0:1789760775.868635:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760775.868685:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565434240:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.880852:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984531 gen 3 00000008:00010000:1.0:1789760775.880915:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760775.880925:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.880938:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.881054:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760775.881090:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565434240:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.881297:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17223, 17223] 00000080:00200000:0.0:1789760775.881315:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.881320:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.881336:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70549504 i_blocks 137816 00000020:00200000:0.0:1789760775.881340:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.881352:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.881445:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70549504, count: 4096 00000080:00200000:0.0:1789760775.881453:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70549504, bytes: 4096 00000080:00200000:0.0:1789760775.881468:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.881477:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.881479:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70549504, 70553600) 0 0 00000080:00200000:0.0:1789760775.881825:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.881844:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17224, 17224] 00020000:00200000:0.0:1789760775.881853:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70549504 chunk: [70549504, 70553600) 70553600, 4096 00020000:00200000:0.0:1789760775.881858:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.881878:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17224, 17224] 00000008:00010000:0.0:1789760775.881897:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.881902:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.881914:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70549504, 70553600) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.881928:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70549504=0x4348000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.881993:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.881996:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70553600 00000020:00010000:0.0:1789760775.882004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.882007:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.882011:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.882014:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.882017:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.882026:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.882033:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.882060:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760775.882192:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 70549504<>70553600 00000100:00100000:1.0:1789760775.882370:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760775.882482:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565434368:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.893307:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984532 gen 3 00000008:00010000:1.0:1789760775.893362:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760775.893371:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.893383:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.893454:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760775.893503:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565434368:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.894312:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17224, 17224] 00000080:00200000:0.0:1789760775.894332:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.894339:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.894356:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70553600 i_blocks 137824 00000020:00200000:0.0:1789760775.894361:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.894374:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.894409:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70553600, count: 4096 00000080:00200000:0.0:1789760775.894416:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70553600, bytes: 4096 00000080:00200000:0.0:1789760775.894430:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.894439:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.894442:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70553600, 70557696) 0 0 00000080:00200000:0.0:1789760775.894447:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.894621:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17225, 17225] 00020000:00200000:0.0:1789760775.894632:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70553600 chunk: [70553600, 70557696) 70557696, 4096 00020000:00200000:0.0:1789760775.894637:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.894679:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17225, 17225] 00000008:00010000:0.0:1789760775.894698:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.894705:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.894717:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70553600, 70557696) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.894731:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70553600=0x4349000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.894800:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.894803:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70557696 00000020:00010000:0.0:1789760775.894812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.894816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.894819:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.894823:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.894826:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.894835:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.894842:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.894867:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760775.895066:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 70553600<>70557696 00000100:00100000:1.0:1789760775.895153:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565434496:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.904160:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984533 gen 3 00000008:00010000:1.0:1789760775.904218:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760775.904226:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.904237:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.904259:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:1.0:1789760775.904290:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565434496:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760775.904337:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17225, 17225] 00000080:00200000:0.0:1789760775.904354:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760775.904360:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760775.904376:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70557696 i_blocks 137832 00000020:00200000:0.0:1789760775.904381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760775.904392:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760775.904427:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70557696, count: 4096 00000080:00200000:0.0:1789760775.904435:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70557696, bytes: 4096 00000080:00200000:0.0:1789760775.904443:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760775.904451:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760775.904453:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70557696, 70561792) 0 0 00000080:00200000:0.0:1789760775.904458:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760775.904485:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17226, 17226] 00020000:00200000:0.0:1789760775.904494:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70557696 chunk: [70557696, 70561792) 70561792, 4096 00020000:00200000:0.0:1789760775.904498:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760775.904536:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17226, 17226] 00000008:00010000:0.0:1789760775.904556:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760775.904562:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760775.904572:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70557696, 70561792) append 0 append lockpos 0 00000080:00200000:0.0:1789760775.904586:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70557696=0x434a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760775.904684:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760775.904688:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70561792 00000020:00010000:0.0:1789760775.904697:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760775.904700:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.904703:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760775.904707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760775.904710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760775.904718:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760775.904725:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760775.904743:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dcc0 00000008:00100000:0.0:1789760775.904869:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42680 - object 0x240000400:5 offset 70557696<>70561792 00000100:00100000:1.0:1789760775.904925:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565434624:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.924359:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984534 gen 3 00000008:00010000:1.0:1789760775.924405:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760775.924413:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.924424:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.924537:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760775.924575:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565434624:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.925425:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565434752:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760775.946539:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17227, 17227] 00000080:00200000:1.0:1789760775.946556:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760775.946562:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760775.946585:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70565888 i_blocks 137848 00000020:00200000:1.0:1789760775.946590:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760775.946601:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760775.947705:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70565888, count: 4096 00000080:00200000:1.0:1789760775.947719:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70565888, bytes: 4096 00000080:00200000:1.0:1789760775.947732:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760775.947741:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760775.947744:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70565888, 70569984) 0 0 00000080:00200000:1.0:1789760775.947749:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760775.947767:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17228, 17228] 00020000:00200000:1.0:1789760775.947778:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70565888 chunk: [70565888, 70569984) 70569984, 4096 00020000:00200000:1.0:1789760775.947783:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760775.947806:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17228, 17228] 00000008:00010000:1.0:1789760775.947826:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760775.947833:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760775.947846:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70565888, 70569984) append 0 append lockpos 0 00000080:00200000:1.0:1789760775.947861:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70565888=0x434c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760775.948174:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760775.948178:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70569984 00000020:00010000:1.0:1789760775.948188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760775.948192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760775.948195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760775.948199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760775.948202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760775.948210:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760775.948217:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760775.948238:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760775.948427:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 70565888<>70569984 00000100:00100000:0.0:1789760775.948641:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565434880:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760775.961876:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984536 gen 3 00000008:00010000:1.0:1789760775.961928:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760775.961937:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760775.961948:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760775.961973:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760775.962004:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565434880:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.963615:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565435008:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.976109:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984537 gen 3 00000008:00010000:3.0:1789760775.976157:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760775.976165:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760775.976175:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760775.976188:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760775.976216:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565435008:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.985429:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760775.985458:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565435136:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760775.997893:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984538 gen 3 00000008:00010000:3.0:1789760775.997956:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760775.997964:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760775.997976:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760775.998001:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760775.998031:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565435136:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.013060:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17231, 17231] 00000080:00200000:1.0:1789760776.013081:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.013087:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.013107:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70582272 i_blocks 137880 00000020:00200000:1.0:1789760776.013111:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.013124:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.013226:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70582272, count: 4096 00000080:00200000:1.0:1789760776.013235:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70582272, bytes: 4096 00000080:00200000:1.0:1789760776.013246:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.013256:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.013259:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70582272, 70586368) 0 0 00000080:00200000:1.0:1789760776.013265:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.013285:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17232, 17232] 00020000:00200000:1.0:1789760776.013297:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70582272 chunk: [70582272, 70586368) 70586368, 4096 00020000:00200000:1.0:1789760776.013303:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.013328:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17232, 17232] 00000008:00010000:1.0:1789760776.013352:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.013358:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.013371:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70582272, 70586368) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.013389:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70582272=0x4350000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.013468:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.013471:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70586368 00000020:00010000:1.0:1789760776.013482:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.013486:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.013489:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.013493:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.013496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.013504:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.013511:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.013537:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760776.013771:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 70582272<>70586368 00000100:00100000:1.0:1789760776.013818:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760776.013836:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565435392:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760776.029098:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984540 gen 3 00000008:00010000:1.0:1789760776.029151:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760776.029161:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760776.029172:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760776.029255:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760776.029287:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565435392:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.036412:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760776.036446:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565435520:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.048206:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984541 gen 3 00000008:00010000:3.0:1789760776.048258:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189e40 00000020:00200000:3.0:1789760776.048267:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.048278:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.048301:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189e40 00000100:00100000:3.0:1789760776.048330:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565435520:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.050184:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565435648:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.065428:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984542 gen 3 00000008:00010000:3.0:1789760776.065477:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441890c0 00000020:00200000:3.0:1789760776.065496:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.065508:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.065542:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441890c0 00000100:00100000:3.0:1789760776.065573:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565435648:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.065814:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17234, 17234] 00000080:00200000:1.0:1789760776.065832:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.065839:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.065865:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70594560 i_blocks 137904 00000020:00200000:1.0:1789760776.065869:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.065879:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.065923:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70594560, count: 4096 00000080:00200000:1.0:1789760776.065930:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70594560, bytes: 4096 00000080:00200000:1.0:1789760776.065941:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.065950:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.065952:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70594560, 70598656) 0 0 00000080:00200000:1.0:1789760776.065957:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.065974:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17235, 17235] 00020000:00200000:1.0:1789760776.065984:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70594560 chunk: [70594560, 70598656) 70598656, 4096 00020000:00200000:1.0:1789760776.065988:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.066007:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17235, 17235] 00000008:00010000:1.0:1789760776.066025:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.066032:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.066044:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70594560, 70598656) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.066059:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70594560=0x4353000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.066129:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.066132:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70598656 00000020:00010000:1.0:1789760776.066140:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.066143:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.066146:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.066150:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.066153:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.066161:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.066168:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.066185:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760776.066312:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 70594560<>70598656 00000100:00100000:0.0:1789760776.066520:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760776.066549:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565435776:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.084421:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984543 gen 3 00000008:00010000:0.0:1789760776.084748:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760776.084757:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.084768:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.084787:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760776.084815:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565435776:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.084964:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17235, 17235] 00000080:00200000:1.0:1789760776.084980:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.084986:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.085001:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70598656 i_blocks 137912 00000020:00200000:1.0:1789760776.085004:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.085013:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.085046:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70598656, count: 4096 00000080:00200000:1.0:1789760776.085053:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70598656, bytes: 4096 00000080:00200000:1.0:1789760776.085062:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.085071:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.085074:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70598656, 70602752) 0 0 00000080:00200000:1.0:1789760776.085078:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.085093:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17236, 17236] 00020000:00200000:1.0:1789760776.085102:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70598656 chunk: [70598656, 70602752) 70602752, 4096 00020000:00200000:1.0:1789760776.085107:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.085127:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17236, 17236] 00000008:00010000:1.0:1789760776.085142:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.085147:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.085157:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70598656, 70602752) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.085169:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70598656=0x4354000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.085219:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.085222:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70602752 00000020:00010000:1.0:1789760776.085828:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.085835:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.085838:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.085841:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.085844:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.085853:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.085860:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.085983:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760776.086117:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 70598656<>70602752 00000100:00100000:0.0:1789760776.086265:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565435904:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.100007:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984544 gen 3 00000008:00010000:0.0:1789760776.100057:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760776.100066:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.100077:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.100105:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760776.100136:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565435904:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760776.100284:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17236, 17236] 00000080:00200000:3.0:1789760776.100339:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760776.100345:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760776.100361:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70602752 i_blocks 137920 00000020:00200000:3.0:1789760776.100365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760776.100373:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760776.100416:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70602752, count: 4096 00000080:00200000:3.0:1789760776.100425:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70602752, bytes: 4096 00000080:00200000:3.0:1789760776.100434:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760776.100442:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760776.100445:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70602752, 70606848) 0 0 00000080:00200000:3.0:1789760776.100450:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760776.101396:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17237, 17237] 00020000:00200000:3.0:1789760776.101414:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70602752 chunk: [70602752, 70606848) 70606848, 4096 00020000:00200000:3.0:1789760776.101419:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760776.101444:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17237, 17237] 00000008:00010000:3.0:1789760776.101965:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760776.101972:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760776.101989:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70602752, 70606848) append 0 append lockpos 0 00000080:00200000:3.0:1789760776.102005:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70602752=0x4355000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760776.102075:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760776.102079:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70606848 00000020:00010000:3.0:1789760776.102087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760776.102090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760776.102094:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760776.102098:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760776.102101:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760776.102112:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760776.102120:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760776.102142:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760776.102859:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 70602752<>70606848 00000100:00100000:3.0:1789760776.102919:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760776.102945:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565436032:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.128722:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984545 gen 3 00000008:00010000:3.0:1789760776.129044:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:3.0:1789760776.129054:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.129065:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.129089:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:3.0:1789760776.129119:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565436032:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.143223:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17238, 17238] 00000080:00200000:0.0:1789760776.143239:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.143245:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.143261:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70610944 i_blocks 137936 00000020:00200000:0.0:1789760776.143264:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.143275:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.143310:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70610944, count: 4096 00000080:00200000:0.0:1789760776.143317:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70610944, bytes: 4096 00000080:00200000:0.0:1789760776.143329:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.143337:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.143339:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70610944, 70615040) 0 0 00000080:00200000:0.0:1789760776.143343:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.143360:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17239, 17239] 00020000:00200000:0.0:1789760776.143369:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70610944 chunk: [70610944, 70615040) 70615040, 4096 00020000:00200000:0.0:1789760776.143373:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.143392:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17239, 17239] 00000008:00010000:0.0:1789760776.143411:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.143417:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.143429:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70610944, 70615040) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.143442:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70610944=0x4357000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.143817:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.143821:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70615040 00000020:00010000:0.0:1789760776.143830:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.143834:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.143837:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.143840:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.143843:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.143851:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.143857:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.143940:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760776.144060:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70610944<>70615040 00000100:00100000:1.0:1789760776.144140:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565436288:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.156745:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984547 gen 3 00000008:00010000:0.0:1789760776.156796:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:0.0:1789760776.156805:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.156816:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.156903:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:0.0:1789760776.156935:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565436288:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.160611:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760776.160639:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565436416:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.173357:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984548 gen 3 00000008:00010000:3.0:1789760776.173415:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760776.173425:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.173437:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.173944:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760776.173974:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565436416:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760776.188497:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17241, 17241] 00000080:00200000:3.0:1789760776.188512:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760776.188518:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760776.188543:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70623232 i_blocks 137960 00000020:00200000:3.0:1789760776.188549:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760776.188559:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760776.188709:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70623232, count: 4096 00000080:00200000:3.0:1789760776.188717:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70623232, bytes: 4096 00000080:00200000:3.0:1789760776.188730:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760776.188740:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760776.188742:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70623232, 70627328) 0 0 00000080:00200000:3.0:1789760776.188747:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760776.188763:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17242, 17242] 00020000:00200000:3.0:1789760776.188772:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70623232 chunk: [70623232, 70627328) 70627328, 4096 00020000:00200000:3.0:1789760776.188776:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760776.188797:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17242, 17242] 00000008:00010000:3.0:1789760776.188816:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760776.188944:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760776.188959:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70623232, 70627328) append 0 append lockpos 0 00000080:00200000:3.0:1789760776.188974:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70623232=0x435a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760776.189033:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760776.189036:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70627328 00000020:00010000:3.0:1789760776.189045:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760776.189048:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760776.189051:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760776.189054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760776.189057:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760776.189066:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760776.189073:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760776.189091:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760776.193310:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 70623232<>70627328 00000100:00100000:3.0:1789760776.193419:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760776.193448:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565436672:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.205513:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984550 gen 3 00000008:00010000:3.0:1789760776.205571:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:3.0:1789760776.205580:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.205737:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.205755:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:3.0:1789760776.205785:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565436672:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.220974:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17243, 17243] 00000080:00200000:0.0:1789760776.220993:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.220999:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.221014:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70631424 i_blocks 137976 00000020:00200000:0.0:1789760776.221019:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.221030:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.221066:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70631424, count: 4096 00000080:00200000:0.0:1789760776.221073:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70631424, bytes: 4096 00000080:00200000:0.0:1789760776.221085:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.221094:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.221096:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70631424, 70635520) 0 0 00000080:00200000:0.0:1789760776.221101:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.221119:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17244, 17244] 00020000:00200000:0.0:1789760776.221129:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70631424 chunk: [70631424, 70635520) 70635520, 4096 00020000:00200000:0.0:1789760776.221134:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.221154:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17244, 17244] 00000008:00010000:0.0:1789760776.221171:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.221178:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.221256:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70631424, 70635520) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.221271:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70631424=0x435c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.221328:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.221331:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70635520 00000020:00010000:0.0:1789760776.221339:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.221343:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.221345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.221349:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.221352:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.221360:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.221367:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.221384:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760776.222335:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70631424<>70635520 00000100:00100000:0.0:1789760776.222414:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565436928:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.232951:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984552 gen 3 00000008:00010000:0.0:1789760776.233002:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:0.0:1789760776.233011:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.233024:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.233092:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:0.0:1789760776.233122:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565436928:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.233427:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17244, 17244] 00000080:00200000:1.0:1789760776.233443:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.233450:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.233578:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70635520 i_blocks 137984 00000020:00200000:1.0:1789760776.233583:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.233595:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.233858:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70635520, count: 4096 00000080:00200000:1.0:1789760776.233867:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70635520, bytes: 4096 00000080:00200000:1.0:1789760776.233932:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.233940:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.233943:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70635520, 70639616) 0 0 00000080:00200000:1.0:1789760776.233947:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.233965:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17245, 17245] 00020000:00200000:1.0:1789760776.233974:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70635520 chunk: [70635520, 70639616) 70639616, 4096 00020000:00200000:1.0:1789760776.233979:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.233998:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17245, 17245] 00000008:00010000:1.0:1789760776.234015:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.234021:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.234034:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70635520, 70639616) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.234048:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70635520=0x435d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.234113:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.234115:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70639616 00000020:00010000:1.0:1789760776.234124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.234127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.234130:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.234134:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.234136:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.234145:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.234151:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.234172:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760776.234302:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 70635520<>70639616 00000100:00100000:0.0:1789760776.234385:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565437056:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.246703:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984553 gen 3 00000008:00010000:0.0:1789760776.246753:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760776.246762:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.246772:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.246784:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760776.246817:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565437056:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.246959:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17245, 17245] 00000080:00200000:1.0:1789760776.246975:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.246981:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.246995:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70639616 i_blocks 137992 00000020:00200000:1.0:1789760776.246999:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.247009:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.247045:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70639616, count: 4096 00000080:00200000:1.0:1789760776.247052:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70639616, bytes: 4096 00000080:00200000:1.0:1789760776.247061:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.247070:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.247073:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70639616, 70643712) 0 0 00000080:00200000:1.0:1789760776.247077:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.247095:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17246, 17246] 00020000:00200000:1.0:1789760776.247104:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70639616 chunk: [70639616, 70643712) 70643712, 4096 00020000:00200000:1.0:1789760776.247108:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.247128:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17246, 17246] 00000008:00010000:1.0:1789760776.247146:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.247153:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.247164:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70639616, 70643712) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.247177:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70639616=0x435e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.247232:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.247235:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70643712 00000020:00010000:1.0:1789760776.247243:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.247247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.247250:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.247253:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.247255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.247262:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.247269:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.247285:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760776.247403:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 70639616<>70643712 00000100:00100000:1.0:1789760776.247452:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565437184:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.257760:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984554 gen 3 00000008:00010000:0.0:1789760776.257813:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760776.257821:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.257834:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.257864:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:3.0:1789760776.258546:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760776.258572:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565437312:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.258742:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565437184:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.273800:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984555 gen 3 00000008:00010000:3.0:1789760776.273848:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760776.273857:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.273868:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.273956:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760776.273989:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565437312:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.275566:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565437440:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.290863:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984556 gen 3 00000008:00010000:3.0:1789760776.290912:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760776.290921:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.290933:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.290960:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760776.290991:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565437440:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.292922:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000080:00200000:0.0:1789760776.303138:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17249, 17249] 00000080:00200000:0.0:1789760776.303154:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.303159:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.303183:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70656000 i_blocks 138024 00000020:00200000:0.0:1789760776.303185:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.303194:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.303232:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70656000, count: 4096 00000080:00200000:0.0:1789760776.303237:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70656000, bytes: 4096 00000080:00200000:0.0:1789760776.303251:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.303258:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.303260:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70656000, 70660096) 0 0 00000080:00200000:0.0:1789760776.303264:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.303278:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17250, 17250] 00020000:00200000:0.0:1789760776.303285:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70656000 chunk: [70656000, 70660096) 70660096, 4096 00020000:00200000:0.0:1789760776.303289:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.303305:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17250, 17250] 00000008:00010000:0.0:1789760776.303320:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.303325:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.303336:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70656000, 70660096) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.303349:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70656000=0x4362000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.303409:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.303412:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70660096 00000020:00010000:0.0:1789760776.303421:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.303425:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.303428:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.303432:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.303435:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.303443:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.303449:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.303466:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760776.304091:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70656000<>70660096 00000100:00100000:0.0:1789760776.304135:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760776.304154:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565437696:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.329280:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984558 gen 3 00000008:00010000:0.0:1789760776.329392:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:0.0:1789760776.329401:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.329413:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.329441:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:0.0:1789760776.329473:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565437696:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.344759:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17251, 17251] 00000080:00200000:0.0:1789760776.344775:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.344780:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.344795:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70664192 i_blocks 138040 00000020:00200000:0.0:1789760776.344798:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.344807:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.344839:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70664192, count: 4096 00000080:00200000:0.0:1789760776.344845:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70664192, bytes: 4096 00000080:00200000:0.0:1789760776.344855:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.344862:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.344863:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70664192, 70668288) 0 0 00000080:00200000:0.0:1789760776.344867:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.344885:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17252, 17252] 00020000:00200000:0.0:1789760776.344895:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70664192 chunk: [70664192, 70668288) 70668288, 4096 00020000:00200000:0.0:1789760776.344899:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.344922:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17252, 17252] 00000008:00010000:0.0:1789760776.344940:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.344946:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.344957:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70664192, 70668288) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.344971:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70664192=0x4364000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.345036:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.345039:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70668288 00000020:00010000:0.0:1789760776.345052:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.345056:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.345059:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.345063:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.345067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.345074:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.345082:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.345099:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760776.345213:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70664192<>70668288 00000100:00100000:1.0:1789760776.345290:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565437952:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.355987:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984560 gen 3 00000008:00010000:0.0:1789760776.356037:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:0.0:1789760776.356045:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.356056:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.356080:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:0.0:1789760776.356111:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565437952:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.360329:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760776.360369:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565438080:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.372227:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984561 gen 3 00000008:00010000:3.0:1789760776.372280:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760776.372288:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.372299:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.372325:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760776.372357:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565438080:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.373323:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565438208:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.387133:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17254, 17254] 00000080:00200000:0.0:1789760776.387146:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.387226:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.387241:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70676480 i_blocks 138064 00000020:00200000:0.0:1789760776.387245:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.387255:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.387285:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70676480, count: 4096 00000080:00200000:0.0:1789760776.387291:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70676480, bytes: 4096 00000080:00200000:0.0:1789760776.387301:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.387308:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.387310:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70676480, 70680576) 0 0 00000080:00200000:0.0:1789760776.387315:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.387328:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17255, 17255] 00020000:00200000:0.0:1789760776.387335:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70676480 chunk: [70676480, 70680576) 70680576, 4096 00020000:00200000:0.0:1789760776.387340:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.387354:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17255, 17255] 00000008:00010000:0.0:1789760776.387366:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.387370:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.387379:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70676480, 70680576) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.387389:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70676480=0x4367000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.387434:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.387437:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70680576 00000020:00010000:0.0:1789760776.387444:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.387447:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.387450:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.387454:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.387456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.387738:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.387744:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.387764:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760776.387997:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70676480<>70680576 00000100:00100000:0.0:1789760776.388041:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565438336:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.400896:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984563 gen 3 00000008:00010000:0.0:1789760776.400932:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:0.0:1789760776.400937:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.400946:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.400968:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:0.0:1789760776.400991:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565438336:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.404369:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17255, 17255] 00000080:00200000:1.0:1789760776.404393:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.404399:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.404412:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70680576 i_blocks 138072 00000020:00200000:1.0:1789760776.404417:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.404431:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.404784:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70680576, count: 4096 00000080:00200000:1.0:1789760776.404793:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70680576, bytes: 4096 00000080:00200000:1.0:1789760776.404803:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.404811:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.404813:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70680576, 70684672) 0 0 00000080:00200000:1.0:1789760776.404817:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.404829:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17256, 17256] 00020000:00200000:1.0:1789760776.404837:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70680576 chunk: [70680576, 70684672) 70684672, 4096 00020000:00200000:1.0:1789760776.404840:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.404856:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17256, 17256] 00000008:00010000:1.0:1789760776.404870:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.404875:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.404885:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70680576, 70684672) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.404897:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70680576=0x4368000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.404947:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.404950:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70684672 00000020:00010000:1.0:1789760776.404957:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.404960:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.404962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.404965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.404968:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.404974:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.404979:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.404998:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760776.405099:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 70680576<>70684672 00000100:00100000:0.0:1789760776.405176:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565438464:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.426339:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984564 gen 3 00000008:00010000:0.0:1789760776.426376:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760776.426383:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.426392:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.426472:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760776.426497:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565438464:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.435798:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17256, 17256] 00000080:00200000:1.0:1789760776.435820:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.435825:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.435877:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70684672 i_blocks 138080 00000020:00200000:1.0:1789760776.435882:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.435892:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.435920:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70684672, count: 4096 00000080:00200000:1.0:1789760776.435927:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70684672, bytes: 4096 00000080:00200000:1.0:1789760776.435938:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.435946:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.435947:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70684672, 70688768) 0 0 00000080:00200000:1.0:1789760776.435951:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.435963:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17257, 17257] 00020000:00200000:1.0:1789760776.435971:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70684672 chunk: [70684672, 70688768) 70688768, 4096 00020000:00200000:1.0:1789760776.435975:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.436001:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17257, 17257] 00000008:00010000:1.0:1789760776.436015:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.436020:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.436030:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70684672, 70688768) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.436041:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70684672=0x4369000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.436086:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.436089:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70688768 00000020:00010000:1.0:1789760776.436096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.436099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.436102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.436105:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.436108:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.436115:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.436120:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.436134:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760776.436235:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 70684672<>70688768 00000100:00100000:0.0:1789760776.436413:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565438592:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.454054:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984565 gen 3 00000008:00010000:0.0:1789760776.454091:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760776.454097:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.454106:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.454126:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760776.454149:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565438592:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.454298:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17257, 17257] 00000080:00200000:1.0:1789760776.454310:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.454316:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.454327:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70688768 i_blocks 138088 00000020:00200000:1.0:1789760776.454331:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.454339:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.454365:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70688768, count: 4096 00000080:00200000:1.0:1789760776.454371:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70688768, bytes: 4096 00000080:00200000:1.0:1789760776.454379:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.454386:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.454388:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70688768, 70692864) 0 0 00000080:00200000:1.0:1789760776.454392:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.454405:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17258, 17258] 00020000:00200000:1.0:1789760776.454411:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70688768 chunk: [70688768, 70692864) 70692864, 4096 00020000:00200000:1.0:1789760776.454416:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.454432:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17258, 17258] 00000008:00010000:1.0:1789760776.454445:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.454450:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.454459:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70688768, 70692864) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.454476:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70688768=0x436a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.454520:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.454532:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70692864 00000020:00010000:1.0:1789760776.454540:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.454543:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.454546:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a908/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.454549:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.454552:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.454559:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.454564:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.454579:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760776.454708:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 70688768<>70692864 00000100:00100000:0.0:1789760776.454865:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760776.454889:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565438720:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.468128:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984566 gen 3 00000008:00010000:0.0:1789760776.468166:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760776.468172:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.468180:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.468247:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760776.468269:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565438720:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.468551:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17258, 17258] 00000080:00200000:1.0:1789760776.468564:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.468568:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760776.469393:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565438848:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.488576:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17259, 17259] 00000080:00200000:0.0:1789760776.488588:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.488593:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.488604:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70696960 i_blocks 138104 00000020:00200000:0.0:1789760776.488607:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.488616:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.488660:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70696960, count: 4096 00000080:00200000:0.0:1789760776.488667:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70696960, bytes: 4096 00000080:00200000:0.0:1789760776.488675:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.488682:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.488684:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70696960, 70701056) 0 0 00000080:00200000:0.0:1789760776.488688:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.488699:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17260, 17260] 00020000:00200000:0.0:1789760776.488706:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70696960 chunk: [70696960, 70701056) 70701056, 4096 00020000:00200000:0.0:1789760776.488709:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.488723:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17260, 17260] 00000008:00010000:0.0:1789760776.488825:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.488831:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.488843:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70696960, 70701056) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.488854:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70696960=0x436c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.488901:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.488904:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70701056 00000020:00010000:0.0:1789760776.488910:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.488914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.488917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.488920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.488922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.488927:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.488932:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.488947:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760776.489115:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70696960<>70701056 00000100:00100000:0.0:1789760776.489159:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760776.489176:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565438976:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.507946:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984568 gen 3 00000008:00010000:0.0:1789760776.507984:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:0.0:1789760776.507991:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.507999:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.508025:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:0.0:1789760776.508049:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565438976:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.532282:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17261, 17261] 00000080:00200000:0.0:1789760776.532301:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.532307:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.532346:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70705152 i_blocks 138120 00000020:00200000:0.0:1789760776.532350:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.532360:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.532386:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70705152, count: 4096 00000080:00200000:0.0:1789760776.532392:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70705152, bytes: 4096 00000080:00200000:0.0:1789760776.532403:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.532412:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.532414:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70705152, 70709248) 0 0 00000080:00200000:0.0:1789760776.532417:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.532430:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17262, 17262] 00020000:00200000:0.0:1789760776.532437:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70705152 chunk: [70705152, 70709248) 70709248, 4096 00020000:00200000:0.0:1789760776.532441:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.532456:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17262, 17262] 00000008:00010000:0.0:1789760776.532469:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.532473:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.532483:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70705152, 70709248) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.532493:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70705152=0x436e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.534747:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.534754:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70709248 00000020:00010000:0.0:1789760776.534766:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.534770:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.534774:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.534777:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.534779:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.534789:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.534805:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.535037:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760776.535154:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 70705152<>70709248 00000100:00100000:1.0:1789760776.537090:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760776.537128:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565439232:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.551061:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984570 gen 3 00000008:00010000:0.0:1789760776.551154:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:0.0:1789760776.551161:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.551170:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.551188:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:0.0:1789760776.551211:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565439232:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.551309:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17262, 17262] 00000080:00200000:1.0:1789760776.551321:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.551326:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.551338:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70709248 i_blocks 138128 00000020:00200000:1.0:1789760776.551341:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.551349:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.551373:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70709248, count: 4096 00000080:00200000:1.0:1789760776.551379:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70709248, bytes: 4096 00000080:00200000:1.0:1789760776.551389:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.551396:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.551398:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70709248, 70713344) 0 0 00000080:00200000:1.0:1789760776.551401:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.551413:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17263, 17263] 00020000:00200000:1.0:1789760776.551419:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70709248 chunk: [70709248, 70713344) 70713344, 4096 00020000:00200000:1.0:1789760776.551423:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.551436:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17263, 17263] 00000008:00010000:1.0:1789760776.551447:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.551451:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.551458:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70709248, 70713344) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.551467:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70709248=0x436f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.551505:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.551508:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70713344 00000020:00010000:1.0:1789760776.551514:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.551517:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.551520:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.551533:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.551536:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.551542:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.551547:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.551561:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760776.551687:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 70709248<>70713344 00000100:00100000:1.0:1789760776.551715:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760776.551730:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565439360:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.571418:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984571 gen 3 00000008:00010000:0.0:1789760776.571454:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760776.571460:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.571468:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.571487:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760776.571510:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565439360:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.571762:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17263, 17263] 00000080:00200000:1.0:1789760776.571774:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.571779:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.571791:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70713344 i_blocks 138136 00000020:00200000:1.0:1789760776.571794:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.571802:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.571829:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70713344, count: 4096 00000080:00200000:1.0:1789760776.571835:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70713344, bytes: 4096 00000080:00200000:1.0:1789760776.571843:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.571850:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.571852:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70713344, 70717440) 0 0 00000080:00200000:1.0:1789760776.571856:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.571868:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17264, 17264] 00020000:00200000:1.0:1789760776.571875:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70713344 chunk: [70713344, 70717440) 70717440, 4096 00020000:00200000:1.0:1789760776.571878:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.571893:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17264, 17264] 00000008:00010000:1.0:1789760776.571906:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.571911:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.571919:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70713344, 70717440) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.571932:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70713344=0x4370000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.572269:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.572273:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70717440 00000020:00010000:1.0:1789760776.572280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.572283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.572287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.572289:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.572292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.572298:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.572302:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.572318:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760776.572410:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 70713344<>70717440 00000100:00100000:0.0:1789760776.572493:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565439488:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.591742:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984572 gen 3 00000008:00010000:0.0:1789760776.591792:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760776.591800:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.591812:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.592247:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760776.592282:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565439488:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.592384:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17264, 17264] 00000080:00200000:1.0:1789760776.592401:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.592407:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.592424:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70717440 i_blocks 138144 00000020:00200000:1.0:1789760776.592428:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.592438:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.592484:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70717440, count: 4096 00000080:00200000:1.0:1789760776.592491:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70717440, bytes: 4096 00000080:00200000:1.0:1789760776.592502:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.592510:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.592512:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70717440, 70721536) 0 0 00000080:00200000:1.0:1789760776.592517:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.594388:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17265, 17265] 00020000:00200000:1.0:1789760776.594404:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70717440 chunk: [70717440, 70721536) 70721536, 4096 00020000:00200000:1.0:1789760776.594409:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.594436:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17265, 17265] 00000008:00010000:1.0:1789760776.594461:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.594467:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.594481:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70717440, 70721536) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.594683:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70717440=0x4371000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.594828:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.594832:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70721536 00000020:00010000:1.0:1789760776.594841:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.594845:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.594848:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.594852:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.594855:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.594863:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.594870:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.594890:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760776.595102:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 70717440<>70721536 00000100:00100000:0.0:1789760776.595194:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565439616:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760776.614803:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984573 gen 3 00000008:00010000:1.0:1789760776.614849:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760776.614858:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760776.614869:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760776.614892:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760776.615443:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565439616:192.168.203.147@tcp:4:dd.0 00000008:00010000:1.0:1789760776.615502:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f8441890c0 00000008:00100000:1.0:1789760776.615635:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 70721536<>70725632 00000100:00100000:1.0:1789760776.615758:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565439744:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760776.637458:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984574 gen 3 00000008:00010000:1.0:1789760776.637510:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441890c0 00000020:00200000:1.0:1789760776.637519:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760776.637542:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760776.637568:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441890c0 00000100:00100000:1.0:1789760776.637600:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565439744:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.638981:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565439872:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.663348:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984575 gen 3 00000080:00200000:1.0:1789760776.665272:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17267, 17267] 00000080:00200000:1.0:1789760776.665288:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.665295:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.665309:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70729728 i_blocks 138168 00000020:00200000:1.0:1789760776.665313:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.665322:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.665356:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70729728, count: 4096 00000080:00200000:1.0:1789760776.665363:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70729728, bytes: 4096 00000080:00200000:1.0:1789760776.665377:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.665387:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.665389:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70729728, 70733824) 0 0 00000080:00200000:1.0:1789760776.665394:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.665411:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17268, 17268] 00020000:00200000:1.0:1789760776.665421:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70729728 chunk: [70729728, 70733824) 70733824, 4096 00020000:00200000:1.0:1789760776.665425:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.665446:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17268, 17268] 00000008:00010000:1.0:1789760776.665475:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.665483:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.665495:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70729728, 70733824) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.665510:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70729728=0x4374000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.665584:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.665587:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70733824 00000020:00010000:1.0:1789760776.665595:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.665599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.665603:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.665606:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.665609:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.665616:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.665622:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.665638:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760776.665901:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 70729728<>70733824 00000100:00100000:1.0:1789760776.666017:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760776.666037:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565440000:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.667161:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565440256:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1789760776.667237:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1789760776.667244:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:af634c28-3647-4f73-bb24-40d7878858fb:2357:1876699565440128:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1789760776.667272:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2357:1876699565440384:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760776.670808:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:af634c28-3647-4f73-bb24-40d7878858fb:2357:1876699565440128:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760776.670837:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2357:1876699565440384:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760776.670850:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565440256:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760776.680396:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984576 gen 3 00000008:00010000:1.0:1789760776.680448:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760776.680457:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760776.680468:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760776.680497:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760776.680543:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565440000:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.697811:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17269, 17269] 00000080:00200000:1.0:1789760776.697830:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.697837:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.697852:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70737920 i_blocks 138184 00000020:00200000:1.0:1789760776.697856:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.698105:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.698145:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70737920, count: 4096 00000080:00200000:1.0:1789760776.698153:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70737920, bytes: 4096 00000080:00200000:1.0:1789760776.698165:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.698173:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.698176:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70737920, 70742016) 0 0 00000080:00200000:1.0:1789760776.698181:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.698197:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17270, 17270] 00020000:00200000:1.0:1789760776.698206:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70737920 chunk: [70737920, 70742016) 70742016, 4096 00020000:00200000:1.0:1789760776.698211:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.698231:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17270, 17270] 00000008:00010000:1.0:1789760776.698249:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.698255:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.698267:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70737920, 70742016) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.698281:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70737920=0x4376000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.698341:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.698345:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70742016 00000020:00010000:1.0:1789760776.698353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.698356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.698359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.698363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.698366:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.698375:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.698384:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.698401:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760776.698795:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 70737920<>70742016 00000100:00100000:0.0:1789760776.699216:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565440640:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760776.715011:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984578 gen 3 00000008:00010000:1.0:1789760776.715293:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760776.715301:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760776.715311:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760776.715458:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760776.715496:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565440640:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.716552:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760776.716571:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565440768:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760776.731268:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1789760776.733608:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17271, 17271] 00000080:00200000:3.0:1789760776.733625:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760776.733632:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760776.734774:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565440896:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.751874:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17272, 17272] 00000080:00200000:0.0:1789760776.751895:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.751901:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.751917:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70750208 i_blocks 138208 00000020:00200000:0.0:1789760776.751922:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.751933:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.751966:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70750208, count: 4096 00000080:00200000:0.0:1789760776.751974:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70750208, bytes: 4096 00000080:00200000:0.0:1789760776.751985:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.751994:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.751997:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70750208, 70754304) 0 0 00000080:00200000:0.0:1789760776.752001:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.752017:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17273, 17273] 00020000:00200000:0.0:1789760776.752026:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70750208 chunk: [70750208, 70754304) 70754304, 4096 00020000:00200000:0.0:1789760776.752032:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.752577:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17273, 17273] 00000008:00010000:0.0:1789760776.752600:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.752606:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.752722:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70750208, 70754304) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.752739:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70750208=0x4379000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.752808:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.752811:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70754304 00000020:00010000:0.0:1789760776.752819:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.752823:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.752825:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.752829:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.752832:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.752839:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.752846:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.752869:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760776.752988:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41f80 - object 0x240000400:5 offset 70750208<>70754304 00000100:00100000:1.0:1789760776.753069:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565441024:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760776.769226:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984581 gen 3 00000008:00010000:1.0:1789760776.769282:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760776.769290:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760776.769301:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760776.769365:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760776.769410:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565441024:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.774064:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17273, 17273] 00000080:00200000:0.0:1789760776.774100:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.774108:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.775061:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70754304 i_blocks 138216 00000020:00200000:0.0:1789760776.775075:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.775100:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.775142:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70754304, count: 4096 00000080:00200000:0.0:1789760776.775153:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70754304, bytes: 4096 00000080:00200000:0.0:1789760776.775169:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.775179:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.775181:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70754304, 70758400) 0 0 00000080:00200000:0.0:1789760776.775187:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.775205:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17274, 17274] 00020000:00200000:0.0:1789760776.775227:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70754304 chunk: [70754304, 70758400) 70758400, 4096 00020000:00200000:0.0:1789760776.775232:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.775254:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17274, 17274] 00000008:00010000:0.0:1789760776.775272:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.775278:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.775290:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70754304, 70758400) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.775305:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70754304=0x437a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.775379:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.775382:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70758400 00000020:00010000:0.0:1789760776.775893:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.775900:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.775904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.775909:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.775912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.775923:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.775932:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.775966:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760776.776111:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 70754304<>70758400 00000100:00100000:1.0:1789760776.776196:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565441152:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760776.791427:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984582 gen 3 00000008:00010000:1.0:1789760776.791481:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760776.791490:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760776.791501:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760776.791595:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760776.791626:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565441152:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.791968:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17274, 17274] 00000080:00200000:0.0:1789760776.791986:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.791992:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760776.792935:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565441280:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.812453:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984583 gen 3 00000008:00010000:3.0:1789760776.812736:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760776.812745:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.812758:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.812786:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760776.812820:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565441280:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.814083:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565441408:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.828825:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984584 gen 3 00000008:00010000:3.0:1789760776.828888:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760776.828897:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.828908:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.828935:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760776.828966:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565441408:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.830444:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565441536:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.853873:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984585 gen 3 00000008:00010000:3.0:1789760776.853933:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760776.853943:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.853955:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.853972:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760776.854004:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565441536:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.854017:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17277, 17277] 00000080:00200000:0.0:1789760776.854035:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.854041:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.854057:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70770688 i_blocks 138248 00000020:00200000:0.0:1789760776.854060:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.854071:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.854106:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70770688, count: 4096 00000080:00200000:0.0:1789760776.854114:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70770688, bytes: 4096 00000080:00200000:0.0:1789760776.854124:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.854133:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.854136:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70770688, 70774784) 0 0 00000080:00200000:0.0:1789760776.854140:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.854157:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17278, 17278] 00020000:00200000:0.0:1789760776.854166:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70770688 chunk: [70770688, 70774784) 70774784, 4096 00020000:00200000:0.0:1789760776.854170:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.854301:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17278, 17278] 00000008:00010000:0.0:1789760776.854320:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.854325:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.854338:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70770688, 70774784) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.854353:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70770688=0x437e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.854428:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.854432:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70774784 00000020:00010000:0.0:1789760776.854440:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.854443:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.854450:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.854453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.854456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.854466:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.854472:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.854497:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760776.854708:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70770688<>70774784 00000100:00100000:1.0:1789760776.854831:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760776.854863:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565441664:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.866957:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984586 gen 3 00000008:00010000:0.0:1789760776.867010:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760776.867020:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.867031:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.867059:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760776.867089:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565441664:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.868158:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565441792:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.877083:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984587 gen 3 00000008:00010000:3.0:1789760776.877137:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760776.877145:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.877157:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.877180:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760776.877210:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565441792:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.878305:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565441920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.890857:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984588 gen 3 00000008:00010000:3.0:1789760776.890911:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760776.890919:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.890930:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.891016:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760776.891050:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565441920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.896070:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565442048:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.905135:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984589 gen 3 00000008:00010000:3.0:1789760776.905194:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760776.905203:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.905215:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.905236:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760776.905268:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565442048:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.907506:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760776.907636:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565442176:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760776.918025:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984590 gen 3 00000008:00010000:3.0:1789760776.918075:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760776.918083:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760776.918094:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760776.918120:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760776.918152:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565442176:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760776.928147:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17283, 17283] 00000080:00200000:0.0:1789760776.928163:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.928169:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.928184:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70795264 i_blocks 138296 00000020:00200000:0.0:1789760776.928187:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.928198:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.928230:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70795264, count: 4096 00000080:00200000:0.0:1789760776.928238:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70795264, bytes: 4096 00000080:00200000:0.0:1789760776.928248:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.928258:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.928261:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70795264, 70799360) 0 0 00000080:00200000:0.0:1789760776.928265:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.928282:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17284, 17284] 00020000:00200000:0.0:1789760776.928291:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70795264 chunk: [70795264, 70799360) 70799360, 4096 00020000:00200000:0.0:1789760776.928296:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.928314:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17284, 17284] 00000008:00010000:0.0:1789760776.928330:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.928336:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.928347:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70795264, 70799360) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.928360:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70795264=0x4384000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.928418:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.928421:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70799360 00000020:00010000:0.0:1789760776.928429:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.928432:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.928435:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.928439:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.928442:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.928449:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.928456:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.928472:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760776.928607:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70795264<>70799360 00000100:00100000:1.0:1789760776.928680:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565442432:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.939489:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984592 gen 3 00000008:00010000:0.0:1789760776.939563:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760776.939573:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.939586:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.939610:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760776.939641:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565442432:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760776.948445:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17285, 17285] 00000080:00200000:3.0:1789760776.948463:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760776.948470:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760776.948485:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70803456 i_blocks 138312 00000020:00200000:3.0:1789760776.948490:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760776.948509:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760776.948586:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70803456, count: 4096 00000080:00200000:3.0:1789760776.948594:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70803456, bytes: 4096 00000080:00200000:3.0:1789760776.948606:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760776.948615:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760776.948618:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70803456, 70807552) 0 0 00000080:00200000:3.0:1789760776.948623:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760776.948639:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17286, 17286] 00020000:00200000:3.0:1789760776.948685:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70803456 chunk: [70803456, 70807552) 70807552, 4096 00020000:00200000:3.0:1789760776.948689:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760776.948708:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17286, 17286] 00000008:00010000:3.0:1789760776.948791:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760776.948798:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760776.948810:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70803456, 70807552) append 0 append lockpos 0 00000080:00200000:3.0:1789760776.948824:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70803456=0x4386000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760776.948878:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760776.948881:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70807552 00000020:00010000:3.0:1789760776.948888:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760776.948891:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760776.948894:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760776.948897:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760776.948900:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760776.948908:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760776.948915:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760776.948932:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760776.949282:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 70803456<>70807552 00000080:00200000:0.0:1789760776.965068:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17286, 17286] 00000080:00200000:0.0:1789760776.965089:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760776.965096:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760776.965112:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70807552 i_blocks 138320 00000020:00200000:0.0:1789760776.965116:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760776.965129:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760776.965166:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70807552, count: 4096 00000080:00200000:0.0:1789760776.965173:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70807552, bytes: 4096 00000080:00200000:0.0:1789760776.965189:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760776.965197:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760776.965200:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70807552, 70811648) 0 0 00000080:00200000:0.0:1789760776.965204:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760776.965221:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17287, 17287] 00020000:00200000:0.0:1789760776.965230:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70807552 chunk: [70807552, 70811648) 70811648, 4096 00020000:00200000:0.0:1789760776.965235:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760776.965255:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17287, 17287] 00000008:00010000:0.0:1789760776.965275:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760776.965782:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760776.965802:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70807552, 70811648) append 0 append lockpos 0 00000080:00200000:0.0:1789760776.965819:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70807552=0x4387000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760776.965909:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760776.965912:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70811648 00000020:00010000:0.0:1789760776.965921:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760776.965925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.965928:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760776.965932:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760776.965935:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760776.965944:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760776.965951:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760776.965982:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760776.966127:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70807552<>70811648 00000100:00100000:0.0:1789760776.966191:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760776.966216:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565442816:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.985479:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984595 gen 3 00000008:00010000:0.0:1789760776.985541:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760776.985551:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.985562:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.985624:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760776.985700:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565442816:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760776.985888:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17287, 17287] 00000080:00200000:1.0:1789760776.985905:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760776.985911:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760776.985928:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70811648 i_blocks 138328 00000020:00200000:1.0:1789760776.985931:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760776.985942:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760776.985980:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70811648, count: 4096 00000080:00200000:1.0:1789760776.985988:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70811648, bytes: 4096 00000080:00200000:1.0:1789760776.986000:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760776.986008:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760776.986011:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70811648, 70815744) 0 0 00000080:00200000:1.0:1789760776.986015:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760776.986034:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17288, 17288] 00020000:00200000:1.0:1789760776.986043:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70811648 chunk: [70811648, 70815744) 70815744, 4096 00020000:00200000:1.0:1789760776.986048:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760776.986069:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17288, 17288] 00000008:00010000:1.0:1789760776.986088:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760776.986094:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760776.986108:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70811648, 70815744) append 0 append lockpos 0 00000080:00200000:1.0:1789760776.986122:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70811648=0x4388000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760776.986225:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760776.986228:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70815744 00000020:00010000:1.0:1789760776.986238:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760776.986241:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.986244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760776.986248:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760776.986251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760776.986259:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760776.986266:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760776.986286:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760776.986427:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 70811648<>70815744 00000100:00100000:1.0:1789760776.986478:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760776.986496:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565442944:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760776.997089:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984596 gen 3 00000008:00010000:0.0:1789760776.997149:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760776.997158:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760776.997168:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760776.997256:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760776.997288:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565442944:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760777.015318:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17289, 17289] 00000080:00200000:3.0:1789760777.015334:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760777.015339:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760777.015353:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70819840 i_blocks 138344 00000020:00200000:3.0:1789760777.015357:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760777.015367:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760777.015399:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70819840, count: 4096 00000080:00200000:3.0:1789760777.015405:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70819840, bytes: 4096 00000080:00200000:3.0:1789760777.015415:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760777.015422:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760777.015424:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70819840, 70823936) 0 0 00000080:00200000:3.0:1789760777.015428:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760777.015443:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17290, 17290] 00020000:00200000:3.0:1789760777.015450:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70819840 chunk: [70819840, 70823936) 70823936, 4096 00020000:00200000:3.0:1789760777.015454:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760777.015623:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17290, 17290] 00000008:00010000:3.0:1789760777.015762:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760777.015769:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760777.015784:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70819840, 70823936) append 0 append lockpos 0 00000080:00200000:3.0:1789760777.015801:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70819840=0x438a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760777.015867:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760777.015871:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70823936 00000020:00010000:3.0:1789760777.015879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760777.015883:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.015886:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.015889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760777.015892:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760777.015900:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760777.015906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760777.015923:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760777.016052:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 70819840<>70823936 00000100:00100000:3.0:1789760777.017217:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760777.017347:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565443200:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.037859:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984598 gen 3 00000008:00010000:3.0:1789760777.037912:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:3.0:1789760777.037920:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760777.037932:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760777.038024:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:3.0:1789760777.038055:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565443200:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760777.057079:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17291, 17291] 00000080:00200000:0.0:1789760777.057098:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.057105:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.057120:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70828032 i_blocks 138360 00000020:00200000:0.0:1789760777.057125:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.057141:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.057173:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70828032, count: 4096 00000080:00200000:0.0:1789760777.057180:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70828032, bytes: 4096 00000080:00200000:0.0:1789760777.057193:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.057202:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.057205:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70828032, 70832128) 0 0 00000080:00200000:0.0:1789760777.057209:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.057226:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17292, 17292] 00020000:00200000:0.0:1789760777.057235:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70828032 chunk: [70828032, 70832128) 70832128, 4096 00020000:00200000:0.0:1789760777.057239:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.057258:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17292, 17292] 00000008:00010000:0.0:1789760777.057275:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.057280:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.057292:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70828032, 70832128) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.057305:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70828032=0x438c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.057364:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.057367:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70832128 00000020:00010000:0.0:1789760777.057374:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.057378:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.057381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.057384:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.057387:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.057394:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.057400:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.058090:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.058223:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70828032<>70832128 00000100:00100000:0.0:1789760777.058282:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565443456:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.074737:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984600 gen 3 00000008:00010000:0.0:1789760777.074789:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.074798:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.074809:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.074831:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.074860:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565443456:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.075324:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17292, 17292] 00000080:00200000:1.0:1789760777.075342:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.075349:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.075365:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70832128 i_blocks 138368 00000020:00200000:1.0:1789760777.075369:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.075382:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.075417:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70832128, count: 4096 00000080:00200000:1.0:1789760777.075424:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70832128, bytes: 4096 00000080:00200000:1.0:1789760777.075435:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.075443:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.075446:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70832128, 70836224) 0 0 00000080:00200000:1.0:1789760777.075450:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.075468:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17293, 17293] 00020000:00200000:1.0:1789760777.075477:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70832128 chunk: [70832128, 70836224) 70836224, 4096 00020000:00200000:1.0:1789760777.075482:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.075502:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17293, 17293] 00000008:00010000:1.0:1789760777.075531:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.075538:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.075549:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70832128, 70836224) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.075563:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70832128=0x438d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.075620:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.075623:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70836224 00000020:00010000:1.0:1789760777.075631:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.075634:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.075637:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.075640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.075688:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.075697:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.075704:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.075721:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760777.075849:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 70832128<>70836224 00000100:00100000:0.0:1789760777.076157:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760777.076241:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565443584:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.092295:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984601 gen 3 00000008:00010000:0.0:1789760777.092345:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760777.092355:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.092366:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.092837:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760777.092870:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565443584:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.095063:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565443712:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.109505:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984602 gen 3 00000008:00010000:3.0:1789760777.109552:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760777.109558:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760777.109567:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760777.109584:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760777.109608:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565443712:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.110482:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565443840:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760777.124924:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17295, 17295] 00000080:00200000:0.0:1789760777.124948:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.124955:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.124972:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70844416 i_blocks 138392 00000020:00200000:0.0:1789760777.124977:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.124990:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.125026:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70844416, count: 4096 00000080:00200000:0.0:1789760777.125034:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70844416, bytes: 4096 00000080:00200000:0.0:1789760777.125049:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.125058:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.125061:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70844416, 70848512) 0 0 00000080:00200000:0.0:1789760777.125066:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.125082:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17296, 17296] 00020000:00200000:0.0:1789760777.125102:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70844416 chunk: [70844416, 70848512) 70848512, 4096 00020000:00200000:0.0:1789760777.125108:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.125130:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17296, 17296] 00000008:00010000:0.0:1789760777.125148:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.125155:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.125167:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70844416, 70848512) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.125190:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70844416=0x4390000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.125546:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.125551:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70848512 00000020:00010000:0.0:1789760777.125561:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.125564:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.125568:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.125572:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.125575:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.125584:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.125591:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.125707:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.125841:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70844416<>70848512 00000100:00100000:1.0:1789760777.127395:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760777.127430:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565443968:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.142436:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984604 gen 3 00000008:00010000:0.0:1789760777.142639:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.142701:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.142712:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.142735:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.143302:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565443968:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.145081:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565444096:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760777.160373:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17297, 17297] 00000080:00200000:0.0:1789760777.160391:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.160396:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.160411:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70852608 i_blocks 138408 00000020:00200000:0.0:1789760777.160415:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.160425:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.160459:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70852608, count: 4096 00000080:00200000:0.0:1789760777.160466:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70852608, bytes: 4096 00000080:00200000:0.0:1789760777.160478:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.160487:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.160490:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70852608, 70856704) 0 0 00000080:00200000:0.0:1789760777.160495:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.160514:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17298, 17298] 00020000:00200000:0.0:1789760777.160632:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70852608 chunk: [70852608, 70856704) 70856704, 4096 00020000:00200000:0.0:1789760777.160638:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.160673:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17298, 17298] 00000008:00010000:0.0:1789760777.160691:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.160696:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.160707:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70852608, 70856704) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.160720:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70852608=0x4392000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.160775:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.160778:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70856704 00000020:00010000:0.0:1789760777.160786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.160795:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.160798:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.160802:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.160805:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.160812:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.160818:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.160838:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.160960:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70852608<>70856704 00000100:00100000:1.0:1789760777.161034:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565444224:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.176817:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984606 gen 3 00000008:00010000:0.0:1789760777.176866:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.176875:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.176885:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.176908:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.176937:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565444224:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760777.205472:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17299, 17299] 00000080:00200000:0.0:1789760777.205488:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.205494:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.205510:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70860800 i_blocks 138424 00000020:00200000:0.0:1789760777.205514:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.205533:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.205572:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70860800, count: 4096 00000080:00200000:0.0:1789760777.205580:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70860800, bytes: 4096 00000080:00200000:0.0:1789760777.205590:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.205600:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.205602:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70860800, 70864896) 0 0 00000080:00200000:0.0:1789760777.205607:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.205623:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17300, 17300] 00020000:00200000:0.0:1789760777.205632:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70860800 chunk: [70860800, 70864896) 70864896, 4096 00020000:00200000:0.0:1789760777.205636:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.205697:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17300, 17300] 00000008:00010000:0.0:1789760777.205715:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.205720:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.205731:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70860800, 70864896) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.205744:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70860800=0x4394000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.205794:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.205796:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70864896 00000020:00010000:0.0:1789760777.205804:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.205808:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.205811:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.205814:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.205817:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.205824:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.205829:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.205844:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.205970:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70860800<>70864896 00000100:00100000:0.0:1789760777.206006:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760777.206023:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565444480:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.216497:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984608 gen 3 00000008:00010000:0.0:1789760777.216558:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.216567:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.216579:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.216603:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.216631:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565444480:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.217496:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565444608:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.230225:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984609 gen 3 00000008:00010000:3.0:1789760777.230281:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760777.230290:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760777.230301:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760777.230320:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760777.230350:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565444608:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.231204:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565444736:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.239899:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984610 gen 3 00000008:00010000:3.0:1789760777.239945:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760777.239954:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760777.240026:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760777.240269:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760777.240303:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565444736:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760777.250844:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17303, 17303] 00000080:00200000:3.0:1789760777.250927:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760777.250935:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760777.250952:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70877184 i_blocks 138456 00000020:00200000:3.0:1789760777.250955:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760777.250965:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760777.251002:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70877184, count: 4096 00000080:00200000:3.0:1789760777.251010:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70877184, bytes: 4096 00000080:00200000:3.0:1789760777.251021:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760777.251029:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760777.251032:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70877184, 70881280) 0 0 00000080:00200000:3.0:1789760777.251036:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760777.251053:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17304, 17304] 00020000:00200000:3.0:1789760777.251061:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70877184 chunk: [70877184, 70881280) 70881280, 4096 00020000:00200000:3.0:1789760777.251066:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760777.251085:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17304, 17304] 00000008:00010000:3.0:1789760777.251104:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760777.251110:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760777.251122:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70877184, 70881280) append 0 append lockpos 0 00000080:00200000:3.0:1789760777.251137:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70877184=0x4398000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760777.251196:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760777.251199:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70881280 00000020:00010000:3.0:1789760777.251207:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760777.251210:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.251213:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.251217:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760777.251220:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760777.251229:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760777.251236:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760777.251254:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760777.251385:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 70877184<>70881280 00000100:00100000:3.0:1789760777.251468:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565444992:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760777.266798:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17304, 17304] 00000080:00200000:3.0:1789760777.266821:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760777.266827:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760777.266844:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70881280 i_blocks 138464 00000020:00200000:3.0:1789760777.266848:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760777.266859:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760777.266894:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70881280, count: 4096 00000080:00200000:3.0:1789760777.266904:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70881280, bytes: 4096 00000080:00200000:3.0:1789760777.266916:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760777.266926:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760777.266929:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70881280, 70885376) 0 0 00000080:00200000:3.0:1789760777.266933:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760777.266952:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17305, 17305] 00020000:00200000:3.0:1789760777.266962:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70881280 chunk: [70881280, 70885376) 70885376, 4096 00020000:00200000:3.0:1789760777.266967:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760777.266990:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17305, 17305] 00000008:00010000:3.0:1789760777.267010:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760777.267016:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760777.267028:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70881280, 70885376) append 0 append lockpos 0 00000080:00200000:3.0:1789760777.267042:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70881280=0x4399000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760777.268800:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760777.268808:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70885376 00000020:00010000:3.0:1789760777.268826:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760777.268830:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.268835:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.268839:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760777.268842:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760777.268855:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760777.268865:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760777.268983:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760777.269136:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 70881280<>70885376 00000100:00100000:3.0:1789760777.269699:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760777.269724:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565445120:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.278788:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984613 gen 3 00000080:00200000:0.0:1789760777.279321:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17305, 17305] 00000080:00200000:0.0:1789760777.279339:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.279345:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.279363:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70885376 i_blocks 138472 00000020:00200000:0.0:1789760777.279368:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.279380:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.279416:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70885376, count: 4096 00000080:00200000:0.0:1789760777.279424:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70885376, bytes: 4096 00000080:00200000:0.0:1789760777.279436:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.279444:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.279446:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70885376, 70889472) 0 0 00000080:00200000:0.0:1789760777.279451:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.279467:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17306, 17306] 00020000:00200000:0.0:1789760777.279477:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70885376 chunk: [70885376, 70889472) 70889472, 4096 00020000:00200000:0.0:1789760777.279482:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.279712:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17306, 17306] 00000008:00010000:0.0:1789760777.279731:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.279737:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.279749:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70885376, 70889472) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.279763:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70885376=0x439a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.279826:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.279829:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70889472 00000020:00010000:0.0:1789760777.279837:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.279917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.279921:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.279925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.279927:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.279935:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.279942:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.279962:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.280092:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70885376<>70889472 00000100:00100000:1.0:1789760777.280244:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565445248:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.293016:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984614 gen 3 00000008:00010000:0.0:1789760777.293070:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.293079:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.293090:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.293116:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.293146:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565445248:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.293263:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17306, 17306] 00000080:00200000:1.0:1789760777.293279:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.293285:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.293301:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70889472 i_blocks 138480 00000020:00200000:1.0:1789760777.293305:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.293316:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.293353:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70889472, count: 4096 00000080:00200000:1.0:1789760777.293359:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70889472, bytes: 4096 00000080:00200000:1.0:1789760777.293369:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.293377:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.293380:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70889472, 70893568) 0 0 00000080:00200000:1.0:1789760777.293385:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.293401:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17307, 17307] 00020000:00200000:1.0:1789760777.293410:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70889472 chunk: [70889472, 70893568) 70893568, 4096 00020000:00200000:1.0:1789760777.293415:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.293434:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17307, 17307] 00000008:00010000:1.0:1789760777.293451:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.293458:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.293471:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70889472, 70893568) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.293485:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70889472=0x439b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.293565:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.293568:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70893568 00000020:00010000:1.0:1789760777.293576:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.293580:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.293583:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.293586:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.293590:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.293598:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.293605:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.293621:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760777.293794:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 70889472<>70893568 00000100:00100000:0.0:1789760777.294440:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760777.294466:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565445376:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.306948:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984615 gen 3 00000008:00010000:0.0:1789760777.307006:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760777.307015:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.307026:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.307045:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760777.307076:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565445376:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.307719:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17307, 17307] 00000080:00200000:1.0:1789760777.307740:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.307746:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.307762:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70893568 i_blocks 138488 00000020:00200000:1.0:1789760777.307766:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.307777:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.307812:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70893568, count: 4096 00000080:00200000:1.0:1789760777.307820:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70893568, bytes: 4096 00000080:00200000:1.0:1789760777.307832:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.307841:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.307844:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70893568, 70897664) 0 0 00000080:00200000:1.0:1789760777.307849:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.307867:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17308, 17308] 00020000:00200000:1.0:1789760777.307876:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70893568 chunk: [70893568, 70897664) 70897664, 4096 00020000:00200000:1.0:1789760777.307881:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.307900:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17308, 17308] 00000008:00010000:1.0:1789760777.307919:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.307926:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.307938:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70893568, 70897664) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.307951:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70893568=0x439c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.308014:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.308017:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70897664 00000020:00010000:1.0:1789760777.308025:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.308029:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.308033:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.308038:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.308041:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.308049:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.308056:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.308074:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760777.308207:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 70893568<>70897664 00000100:00100000:0.0:1789760777.308287:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565445504:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.318009:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984616 gen 3 00000008:00010000:0.0:1789760777.318060:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760777.318069:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.318155:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.318176:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760777.318205:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565445504:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.318322:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17308, 17308] 00000080:00200000:1.0:1789760777.318339:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.318345:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.318361:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70897664 i_blocks 138496 00000020:00200000:1.0:1789760777.318365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.318375:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.318409:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70897664, count: 4096 00000080:00200000:1.0:1789760777.318417:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70897664, bytes: 4096 00000080:00200000:1.0:1789760777.318426:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.318435:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.318438:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70897664, 70901760) 0 0 00000080:00200000:1.0:1789760777.318442:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.318531:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17309, 17309] 00020000:00200000:1.0:1789760777.318543:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70897664 chunk: [70897664, 70901760) 70901760, 4096 00020000:00200000:1.0:1789760777.318548:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.318571:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17309, 17309] 00000008:00010000:1.0:1789760777.318591:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.318597:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.318609:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70897664, 70901760) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.318623:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70897664=0x439d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.319393:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.319399:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70901760 00000020:00010000:1.0:1789760777.319413:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.319417:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.319422:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.319426:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.319429:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.319439:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.319446:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.320348:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760777.320820:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 70897664<>70901760 00000100:00100000:0.0:1789760777.320951:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760777.321050:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565445632:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.332470:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984617 gen 3 00000008:00010000:0.0:1789760777.332533:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760777.332542:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.332555:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.332692:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760777.332729:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565445632:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760777.342813:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17310, 17310] 00000080:00200000:3.0:1789760777.343260:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760777.343268:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760777.343284:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70905856 i_blocks 138512 00000020:00200000:3.0:1789760777.343288:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760777.343302:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760777.343339:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70905856, count: 4096 00000080:00200000:3.0:1789760777.343347:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70905856, bytes: 4096 00000080:00200000:3.0:1789760777.343360:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760777.343370:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760777.343373:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70905856, 70909952) 0 0 00000080:00200000:3.0:1789760777.343377:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760777.343393:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17311, 17311] 00020000:00200000:3.0:1789760777.343402:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70905856 chunk: [70905856, 70909952) 70909952, 4096 00020000:00200000:3.0:1789760777.343406:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760777.343426:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17311, 17311] 00000008:00010000:3.0:1789760777.343445:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760777.343451:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760777.343464:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70905856, 70909952) append 0 append lockpos 0 00000080:00200000:3.0:1789760777.343479:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70905856=0x439f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760777.343559:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760777.343562:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70909952 00000020:00010000:3.0:1789760777.343570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760777.343574:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.343577:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.343581:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760777.343584:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760777.343593:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760777.343600:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760777.343623:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760777.344158:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 70905856<>70909952 00000080:00200000:0.0:1789760777.355744:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17311, 17311] 00000080:00200000:0.0:1789760777.355763:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.355769:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.355785:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70909952 i_blocks 138520 00000020:00200000:0.0:1789760777.355789:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.355801:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.355851:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70909952, count: 4096 00000080:00200000:0.0:1789760777.355858:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70909952, bytes: 4096 00000080:00200000:0.0:1789760777.355879:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.355888:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.355891:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70909952, 70914048) 0 0 00000080:00200000:0.0:1789760777.355896:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.355913:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17312, 17312] 00020000:00200000:0.0:1789760777.355923:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70909952 chunk: [70909952, 70914048) 70914048, 4096 00020000:00200000:0.0:1789760777.355928:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.355949:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17312, 17312] 00000008:00010000:0.0:1789760777.355967:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.355974:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.355986:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70909952, 70914048) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.356000:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70909952=0x43a0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.356063:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.356066:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70914048 00000020:00010000:0.0:1789760777.356091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.356096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.356099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.356102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.356105:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.356113:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.356119:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.356136:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.356270:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70909952<>70914048 00000100:00100000:1.0:1789760777.356370:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565446016:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.368574:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984620 gen 3 00000008:00010000:0.0:1789760777.368625:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.368634:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.368668:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.368760:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.368792:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565446016:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.372820:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565446144:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.390354:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984621 gen 3 00000008:00010000:3.0:1789760777.390406:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760777.390416:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760777.390428:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760777.390530:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760777.390562:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565446144:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760777.391108:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17313, 17313] 00000080:00200000:0.0:1789760777.391127:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.391133:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.391149:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70918144 i_blocks 138536 00000020:00200000:0.0:1789760777.391154:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.391165:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.391200:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70918144, count: 4096 00000080:00200000:0.0:1789760777.391208:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70918144, bytes: 4096 00000080:00200000:0.0:1789760777.391220:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.391229:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.391231:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70918144, 70922240) 0 0 00000080:00200000:0.0:1789760777.391236:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.391254:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17314, 17314] 00020000:00200000:0.0:1789760777.391263:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70918144 chunk: [70918144, 70922240) 70922240, 4096 00020000:00200000:0.0:1789760777.391268:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.391288:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17314, 17314] 00000008:00010000:0.0:1789760777.391309:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.391315:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.391328:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70918144, 70922240) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.391342:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70918144=0x43a2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.391403:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.391406:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70922240 00000020:00010000:0.0:1789760777.391414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.391418:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.391422:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.391425:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.391428:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.391436:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.391442:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.391474:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.391613:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70918144<>70922240 00000100:00100000:1.0:1789760777.391699:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565446272:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.403725:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984622 gen 3 00000008:00010000:0.0:1789760777.403778:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.403786:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.403798:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.403823:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.403852:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565446272:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760777.403964:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17314, 17314] 00000080:00200000:3.0:1789760777.403981:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760777.403988:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760777.404003:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70922240 i_blocks 138544 00000020:00200000:3.0:1789760777.404007:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760777.404018:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760777.404054:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70922240, count: 4096 00000080:00200000:3.0:1789760777.404062:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70922240, bytes: 4096 00000080:00200000:3.0:1789760777.404072:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760777.404080:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760777.404083:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70922240, 70926336) 0 0 00000080:00200000:3.0:1789760777.404087:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760777.404104:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17315, 17315] 00020000:00200000:3.0:1789760777.404113:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70922240 chunk: [70922240, 70926336) 70926336, 4096 00020000:00200000:3.0:1789760777.404118:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760777.404137:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17315, 17315] 00000008:00010000:3.0:1789760777.404156:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760777.404162:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760777.404174:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70922240, 70926336) append 0 append lockpos 0 00000080:00200000:3.0:1789760777.404189:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70922240=0x43a3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760777.404251:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760777.404254:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70926336 00000020:00010000:3.0:1789760777.404262:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760777.404266:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.404269:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.404272:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760777.404275:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760777.404284:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760777.404291:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760777.404307:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760777.404439:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 70922240<>70926336 00000080:00200000:0.0:1789760777.421048:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17315, 17315] 00000080:00200000:0.0:1789760777.421065:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.421071:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.421094:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70926336 i_blocks 138552 00000020:00200000:0.0:1789760777.421097:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.421107:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.421140:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70926336, count: 4096 00000080:00200000:0.0:1789760777.421146:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70926336, bytes: 4096 00000080:00200000:0.0:1789760777.421156:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.421164:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.421167:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70926336, 70930432) 0 0 00000080:00200000:0.0:1789760777.421171:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.421187:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17316, 17316] 00020000:00200000:0.0:1789760777.421195:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70926336 chunk: [70926336, 70930432) 70930432, 4096 00020000:00200000:0.0:1789760777.421201:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.421222:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17316, 17316] 00000008:00010000:0.0:1789760777.421239:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.421245:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.421258:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70926336, 70930432) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.421272:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70926336=0x43a4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.421339:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.421343:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70930432 00000020:00010000:0.0:1789760777.421351:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.421355:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.421358:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.421361:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.421365:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.421373:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.421381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.421397:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.421533:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70926336<>70930432 00000100:00100000:0.0:1789760777.421588:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760777.421607:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565446528:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.432117:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984624 gen 3 00000008:00010000:0.0:1789760777.432169:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.432178:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.432190:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.432213:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.432244:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565446528:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.432272:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17316, 17316] 00000080:00200000:1.0:1789760777.432292:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.432299:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.432316:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70930432 i_blocks 138560 00000020:00200000:1.0:1789760777.432321:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.432333:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.432372:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70930432, count: 4096 00000080:00200000:1.0:1789760777.432380:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70930432, bytes: 4096 00000080:00200000:1.0:1789760777.432391:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.432400:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.432403:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70930432, 70934528) 0 0 00000080:00200000:1.0:1789760777.432407:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.432425:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17317, 17317] 00020000:00200000:1.0:1789760777.432434:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70930432 chunk: [70930432, 70934528) 70934528, 4096 00020000:00200000:1.0:1789760777.432438:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.432469:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17317, 17317] 00000008:00010000:1.0:1789760777.432485:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.432491:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.432502:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70930432, 70934528) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.432513:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70930432=0x43a5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.432582:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.432585:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70934528 00000020:00010000:1.0:1789760777.432593:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.432610:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.432613:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.432617:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.432620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.432627:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.432634:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.432684:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760777.432808:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 70930432<>70934528 00000100:00100000:1.0:1789760777.432849:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760777.433010:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565446656:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.444606:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984625 gen 3 00000008:00010000:0.0:1789760777.444691:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760777.444700:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.444712:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.444744:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760777.444776:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565446656:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760777.464788:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17318, 17318] 00000080:00200000:3.0:1789760777.464806:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760777.464812:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760777.464829:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70938624 i_blocks 138576 00000020:00200000:3.0:1789760777.464832:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760777.464842:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760777.464879:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70938624, count: 4096 00000080:00200000:3.0:1789760777.464887:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70938624, bytes: 4096 00000080:00200000:3.0:1789760777.464899:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760777.464908:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760777.464910:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70938624, 70942720) 0 0 00000080:00200000:3.0:1789760777.464915:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760777.464934:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17319, 17319] 00020000:00200000:3.0:1789760777.464944:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70938624 chunk: [70938624, 70942720) 70942720, 4096 00020000:00200000:3.0:1789760777.464949:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760777.464969:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17319, 17319] 00000008:00010000:3.0:1789760777.464986:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760777.464993:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760777.465005:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70938624, 70942720) append 0 append lockpos 0 00000080:00200000:3.0:1789760777.465019:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70938624=0x43a7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760777.465081:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760777.465084:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70942720 00000020:00010000:3.0:1789760777.465092:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760777.465096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.465099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.465102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760777.465106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760777.465114:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760777.465121:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760777.465138:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760777.465404:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 70938624<>70942720 00000080:00200000:3.0:1789760777.484471:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17319, 17319] 00000080:00200000:3.0:1789760777.484496:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760777.484503:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760777.484526:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70942720 i_blocks 138584 00000020:00200000:3.0:1789760777.484530:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760777.484541:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760777.484579:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70942720, count: 4096 00000080:00200000:3.0:1789760777.484587:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70942720, bytes: 4096 00000080:00200000:3.0:1789760777.484598:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760777.484607:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760777.484609:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70942720, 70946816) 0 0 00000080:00200000:3.0:1789760777.484613:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760777.484632:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17320, 17320] 00020000:00200000:3.0:1789760777.484641:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70942720 chunk: [70942720, 70946816) 70946816, 4096 00020000:00200000:3.0:1789760777.484673:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760777.484693:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17320, 17320] 00000008:00010000:3.0:1789760777.484713:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760777.484720:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760777.484732:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70942720, 70946816) append 0 append lockpos 0 00000080:00200000:3.0:1789760777.484747:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70942720=0x43a8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760777.484806:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760777.484809:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70946816 00000020:00010000:3.0:1789760777.484817:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760777.484821:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.484824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.484828:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760777.484831:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760777.484839:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760777.484846:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760777.485097:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760777.485735:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 70942720<>70946816 00000080:00200000:3.0:1789760777.496328:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17320, 17320] 00000080:00200000:3.0:1789760777.496348:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760777.496355:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760777.496373:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70946816 i_blocks 138592 00000020:00200000:3.0:1789760777.496377:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760777.496388:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760777.496429:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70946816, count: 4096 00000080:00200000:3.0:1789760777.496437:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70946816, bytes: 4096 00000080:00200000:3.0:1789760777.496448:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760777.496457:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760777.496460:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70946816, 70950912) 0 0 00000080:00200000:3.0:1789760777.496465:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760777.496486:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17321, 17321] 00020000:00200000:3.0:1789760777.496495:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70946816 chunk: [70946816, 70950912) 70950912, 4096 00020000:00200000:3.0:1789760777.496500:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760777.496534:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17321, 17321] 00000008:00010000:3.0:1789760777.496552:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760777.496558:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760777.496571:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70946816, 70950912) append 0 append lockpos 0 00000080:00200000:3.0:1789760777.496586:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70946816=0x43a9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760777.496670:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760777.496672:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70950912 00000020:00010000:3.0:1789760777.496682:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760777.496686:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.496689:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760777.496693:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760777.496696:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760777.496705:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760777.496713:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760777.499510:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760777.499715:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 70946816<>70950912 00000080:00200000:3.0:1789760777.514495:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17321, 17321] 00000080:00200000:3.0:1789760777.514530:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760777.514537:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.532261:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17322, 17322] 00000080:00200000:0.0:1789760777.532303:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.532310:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.532331:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70955008 i_blocks 138608 00000020:00200000:0.0:1789760777.532338:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.532357:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.532398:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70955008, count: 4096 00000080:00200000:0.0:1789760777.532407:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70955008, bytes: 4096 00000080:00200000:0.0:1789760777.532422:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.532432:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.532435:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70955008, 70959104) 0 0 00000080:00200000:0.0:1789760777.532439:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.532458:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17323, 17323] 00020000:00200000:0.0:1789760777.532467:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70955008 chunk: [70955008, 70959104) 70959104, 4096 00020000:00200000:0.0:1789760777.532472:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.532495:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17323, 17323] 00000008:00010000:0.0:1789760777.532524:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.532530:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.532543:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70955008, 70959104) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.532557:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70955008=0x43ab000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.532632:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.532636:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70959104 00000020:00010000:0.0:1789760777.532812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.532817:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.532821:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.532825:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.532828:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.532839:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.532847:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.532877:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.533019:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70955008<>70959104 00000100:00100000:1.0:1789760777.533102:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565447424:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.541342:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984631 gen 3 00000008:00010000:0.0:1789760777.541394:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.541403:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.541415:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.541443:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.541984:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565447424:192.168.203.147@tcp:4:dd.0 00000008:00010000:0.0:1789760777.543375:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189a80 00000008:00100000:0.0:1789760777.543773:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70959104<>70963200 00000100:00100000:1.0:1789760777.543844:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760777.544005:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565447552:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.560242:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984632 gen 3 00000008:00010000:0.0:1789760777.560290:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:0.0:1789760777.560298:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.560309:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.560403:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:0.0:1789760777.560435:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565447552:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.561855:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565447680:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760777.570704:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17325, 17325] 00000080:00200000:0.0:1789760777.570726:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.570733:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.570750:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70967296 i_blocks 138632 00000020:00200000:0.0:1789760777.570755:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.570770:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.570804:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70967296, count: 4096 00000080:00200000:0.0:1789760777.570810:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70967296, bytes: 4096 00000080:00200000:0.0:1789760777.570823:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.570832:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.570835:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70967296, 70971392) 0 0 00000080:00200000:0.0:1789760777.570839:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.570855:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17326, 17326] 00020000:00200000:0.0:1789760777.570863:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70967296 chunk: [70967296, 70971392) 70971392, 4096 00020000:00200000:0.0:1789760777.570868:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.570887:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17326, 17326] 00000008:00010000:0.0:1789760777.570905:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.570910:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.570921:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70967296, 70971392) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.570935:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70967296=0x43ae000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.570999:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.571002:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70971392 00000020:00010000:0.0:1789760777.571010:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.571014:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.571017:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.571020:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.571024:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.571031:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.571037:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.571058:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.571189:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70967296<>70971392 00000100:00100000:1.0:1789760777.571318:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565447808:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.586845:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984634 gen 3 00000008:00010000:0.0:1789760777.586957:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.586966:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.586977:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.587064:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.587094:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565447808:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760777.600209:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17327, 17327] 00000080:00200000:0.0:1789760777.600227:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.600233:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.600249:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70975488 i_blocks 138648 00000020:00200000:0.0:1789760777.600254:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.600265:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.600299:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70975488, count: 4096 00000080:00200000:0.0:1789760777.600306:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70975488, bytes: 4096 00000080:00200000:0.0:1789760777.600320:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.600329:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.600332:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70975488, 70979584) 0 0 00000080:00200000:0.0:1789760777.600336:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.600353:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17328, 17328] 00020000:00200000:0.0:1789760777.600362:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70975488 chunk: [70975488, 70979584) 70979584, 4096 00020000:00200000:0.0:1789760777.600372:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.600395:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17328, 17328] 00000008:00010000:0.0:1789760777.600413:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.600419:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7d5b0, osc lock ffff93f850af0000, flags 0x0 00000080:00200000:0.0:1789760777.600430:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70975488, 70979584) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.600443:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70975488=0x43b0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.600970:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.600974:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70979584 00000020:00010000:1.0:1789760777.600985:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.600988:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.600991:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7d5b0 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.600995:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.600998:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7d5b0 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.601007:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.601014:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.601029:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760777.601160:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 70975488<>70979584 00000100:00100000:0.0:1789760777.601221:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565448064:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.621747:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984636 gen 3 00000008:00010000:0.0:1789760777.621804:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760777.621813:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.621825:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.621861:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760777.621896:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565448064:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.623755:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565448192:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.634878:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984637 gen 3 00000008:00010000:3.0:1789760777.634953:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760777.634961:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760777.634975:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760777.635002:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760777.635027:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565448192:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760777.636235:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:3.0:1789760777.636367:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 70983680<>70987776 00000100:00100000:3.0:1789760777.636418:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760777.636436:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565448320:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.647995:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984638 gen 3 00000008:00010000:3.0:1789760777.648046:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760777.648056:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760777.648068:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760777.648100:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760777.648130:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565448320:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.648976:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565448448:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760777.675897:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17331, 17331] 00000080:00200000:0.0:1789760777.675916:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.675923:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.675939:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70991872 i_blocks 138680 00000020:00200000:0.0:1789760777.675943:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.675954:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.675985:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70991872, count: 4096 00000080:00200000:0.0:1789760777.675992:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70991872, bytes: 4096 00000080:00200000:0.0:1789760777.676004:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.676013:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.676015:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70991872, 70995968) 0 0 00000080:00200000:0.0:1789760777.676019:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.676036:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17332, 17332] 00020000:00200000:0.0:1789760777.676044:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70991872 chunk: [70991872, 70995968) 70995968, 4096 00020000:00200000:0.0:1789760777.676048:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.676066:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17332, 17332] 00000008:00010000:0.0:1789760777.676142:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.676148:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760777.676160:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70991872, 70995968) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.676173:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70991872=0x43b4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.676231:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.676233:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 70995968 00000020:00010000:0.0:1789760777.676240:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.676244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.676247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.676251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.676253:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.676261:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.676267:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.676284:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.676398:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 70991872<>70995968 00000100:00100000:1.0:1789760777.676472:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565448576:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.687531:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984640 gen 3 00000008:00010000:0.0:1789760777.687597:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:0.0:1789760777.687608:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.687629:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.687775:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:0.0:1789760777.687805:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565448576:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.688494:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17332, 17332] 00000080:00200000:1.0:1789760777.688586:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.688594:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.688613:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 70995968 i_blocks 138688 00000020:00200000:1.0:1789760777.688617:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.688631:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.689080:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 70995968, count: 4096 00000080:00200000:1.0:1789760777.689092:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 70995968, bytes: 4096 00000080:00200000:1.0:1789760777.689104:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.689114:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.689117:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [70995968, 71000064) 0 0 00000080:00200000:1.0:1789760777.689122:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.689139:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17333, 17333] 00020000:00200000:1.0:1789760777.689148:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 70995968 chunk: [70995968, 71000064) 71000064, 4096 00020000:00200000:1.0:1789760777.689153:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.689174:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17333, 17333] 00000008:00010000:1.0:1789760777.689193:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.689198:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.689223:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [70995968, 71000064) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.689238:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=70995968=0x43b5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.689305:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.689309:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71000064 00000020:00010000:1.0:1789760777.689317:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.689320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.689323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.689327:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.689330:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.689338:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.689345:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.689371:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760777.689498:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 70995968<>71000064 00000100:00100000:0.0:1789760777.689680:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760777.689702:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565448704:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.700839:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984641 gen 3 00000008:00010000:0.0:1789760777.700872:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760777.700878:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.700887:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.700905:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760777.700927:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565448704:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.701022:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17333, 17333] 00000080:00200000:1.0:1789760777.701036:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.701041:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.701054:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71000064 i_blocks 138696 00000020:00200000:1.0:1789760777.701057:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.701067:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.701093:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71000064, count: 4096 00000080:00200000:1.0:1789760777.701099:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71000064, bytes: 4096 00000080:00200000:1.0:1789760777.701107:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.701115:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.701117:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71000064, 71004160) 0 0 00000080:00200000:1.0:1789760777.701120:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.701133:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17334, 17334] 00020000:00200000:1.0:1789760777.701139:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71000064 chunk: [71000064, 71004160) 71004160, 4096 00020000:00200000:1.0:1789760777.701143:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.701158:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17334, 17334] 00000008:00010000:1.0:1789760777.701172:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.701177:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.701185:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71000064, 71004160) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.701195:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71000064=0x43b6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.701235:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.701238:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71004160 00000020:00010000:1.0:1789760777.701244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.701247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.701250:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.701253:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.701256:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.701261:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.701267:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.701280:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760777.701370:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 71000064<>71004160 00000100:00100000:1.0:1789760777.701416:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760777.701429:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565448832:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.710141:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984642 gen 3 00000008:00010000:0.0:1789760777.710177:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760777.710184:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.710192:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.710352:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760777.710380:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565448832:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.710738:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17334, 17334] 00000080:00200000:1.0:1789760777.710754:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.710760:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.710772:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71004160 i_blocks 138704 00000020:00200000:1.0:1789760777.710776:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.710783:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.710808:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71004160, count: 4096 00000080:00200000:1.0:1789760777.710814:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71004160, bytes: 4096 00000080:00200000:1.0:1789760777.710822:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.710829:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.710831:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71004160, 71008256) 0 0 00000080:00200000:1.0:1789760777.710835:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.710846:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17335, 17335] 00020000:00200000:1.0:1789760777.710853:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71004160 chunk: [71004160, 71008256) 71008256, 4096 00020000:00200000:1.0:1789760777.710857:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.710946:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17335, 17335] 00000008:00010000:1.0:1789760777.710960:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.710965:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.710973:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71004160, 71008256) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.710984:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71004160=0x43b7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.711045:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.711048:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71008256 00000020:00010000:1.0:1789760777.711054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.711058:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.711061:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.711063:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.711066:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.711073:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.711078:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.711094:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760777.711191:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 71004160<>71008256 00000100:00100000:0.0:1789760777.711288:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565448960:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.721244:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984643 gen 3 00000008:00010000:0.0:1789760777.721276:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760777.721281:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.721289:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.721304:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760777.721324:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565448960:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.721528:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17335, 17335] 00000080:00200000:1.0:1789760777.721540:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.721545:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.721557:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71008256 i_blocks 138712 00000020:00200000:1.0:1789760777.721561:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.721568:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.721592:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71008256, count: 4096 00000080:00200000:1.0:1789760777.721599:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71008256, bytes: 4096 00000080:00200000:1.0:1789760777.721607:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.721614:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.721616:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71008256, 71012352) 0 0 00000080:00200000:1.0:1789760777.721620:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.721634:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17336, 17336] 00020000:00200000:1.0:1789760777.721641:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71008256 chunk: [71008256, 71012352) 71012352, 4096 00020000:00200000:1.0:1789760777.721679:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.721694:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17336, 17336] 00000008:00010000:1.0:1789760777.721706:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.721711:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.721721:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71008256, 71012352) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.721731:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71008256=0x43b8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.721774:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.721776:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71012352 00000020:00010000:1.0:1789760777.721783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.721787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.721790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.721793:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.721796:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.721802:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.721808:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.721822:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760777.721913:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 71008256<>71012352 00000100:00100000:0.0:1789760777.721986:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760777.722010:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565449088:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.738952:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984644 gen 3 00000008:00010000:0.0:1789760777.738989:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760777.738996:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.739004:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.739023:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760777.739046:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565449088:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.739708:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17336, 17336] 00000080:00200000:1.0:1789760777.739722:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.739728:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.739740:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71012352 i_blocks 138720 00000020:00200000:1.0:1789760777.739744:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.739752:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.739777:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71012352, count: 4096 00000080:00200000:1.0:1789760777.739783:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71012352, bytes: 4096 00000080:00200000:1.0:1789760777.739791:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.739798:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.739800:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71012352, 71016448) 0 0 00000080:00200000:1.0:1789760777.739804:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.739816:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17337, 17337] 00020000:00200000:1.0:1789760777.739823:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71012352 chunk: [71012352, 71016448) 71016448, 4096 00020000:00200000:1.0:1789760777.739827:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.739842:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17337, 17337] 00000008:00010000:1.0:1789760777.739853:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.739857:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.739866:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71012352, 71016448) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.739877:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71012352=0x43b9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.740369:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.740374:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71016448 00000020:00010000:1.0:1789760777.740381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.740384:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.740387:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.740391:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.740393:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.740401:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.740407:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.740425:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760777.741223:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 71012352<>71016448 00000100:00100000:0.0:1789760777.741671:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760777.741695:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565449216:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.749274:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984645 gen 3 00000008:00010000:0.0:1789760777.749327:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760777.749335:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.749347:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.749412:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760777.749441:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565449216:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.750185:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17337, 17337] 00000080:00200000:1.0:1789760777.750206:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.750213:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.750230:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71016448 i_blocks 138728 00000020:00200000:1.0:1789760777.750234:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.750246:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.750280:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71016448, count: 4096 00000080:00200000:1.0:1789760777.750289:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71016448, bytes: 4096 00000080:00200000:1.0:1789760777.750303:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.750313:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.750316:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71016448, 71020544) 0 0 00000080:00200000:1.0:1789760777.750321:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.750339:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17338, 17338] 00020000:00200000:1.0:1789760777.750349:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71016448 chunk: [71016448, 71020544) 71020544, 4096 00020000:00200000:1.0:1789760777.750355:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.750377:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17338, 17338] 00000008:00010000:1.0:1789760777.750397:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.750405:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.750417:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71016448, 71020544) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.750431:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71016448=0x43ba000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.750566:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.750569:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71020544 00000020:00010000:1.0:1789760777.750640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.750708:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.750712:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.750716:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.750719:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.750728:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.750736:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.750760:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760777.751015:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 71016448<>71020544 00000100:00100000:0.0:1789760777.751088:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565449344:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760777.754834:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760777.760287:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984646 gen 3 00000008:00010000:0.0:1789760777.760338:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760777.760347:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.760358:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.760418:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760777.760448:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565449344:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.760794:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17338, 17338] 00000080:00200000:1.0:1789760777.760810:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.760817:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.760832:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71020544 i_blocks 138736 00000020:00200000:1.0:1789760777.760836:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.760846:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.761000:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71020544, count: 4096 00000080:00200000:1.0:1789760777.761009:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71020544, bytes: 4096 00000080:00200000:1.0:1789760777.761021:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.761031:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.761034:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71020544, 71024640) 0 0 00000080:00200000:1.0:1789760777.761038:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.761056:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17339, 17339] 00020000:00200000:1.0:1789760777.761064:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71020544 chunk: [71020544, 71024640) 71024640, 4096 00020000:00200000:1.0:1789760777.761069:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.761089:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17339, 17339] 00000008:00010000:1.0:1789760777.761105:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.761111:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.761125:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71020544, 71024640) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.761139:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71020544=0x43bb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.761199:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.761202:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71024640 00000020:00010000:1.0:1789760777.761211:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.761215:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.761218:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.761222:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.761226:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.761234:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.761241:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.761259:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760777.761384:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 71020544<>71024640 00000100:00100000:0.0:1789760777.761463:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565449472:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.772367:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984647 gen 3 00000008:00010000:0.0:1789760777.772416:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760777.772424:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.772434:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.772569:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760777.772707:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565449472:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.772995:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17339, 17339] 00000080:00200000:1.0:1789760777.773015:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.773021:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.773038:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71024640 i_blocks 138744 00000020:00200000:1.0:1789760777.773043:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.773053:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.773091:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71024640, count: 4096 00000080:00200000:1.0:1789760777.773099:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71024640, bytes: 4096 00000080:00200000:1.0:1789760777.773112:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.773122:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.773125:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71024640, 71028736) 0 0 00000080:00200000:1.0:1789760777.773130:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.773148:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17340, 17340] 00020000:00200000:1.0:1789760777.773158:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71024640 chunk: [71024640, 71028736) 71028736, 4096 00020000:00200000:1.0:1789760777.773163:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.773426:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17340, 17340] 00000008:00010000:1.0:1789760777.773446:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.773573:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.773685:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71024640, 71028736) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.773701:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71024640=0x43bc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.773767:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.773769:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71028736 00000020:00010000:1.0:1789760777.773777:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.773781:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.773784:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.773788:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.773791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.773799:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.773806:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.773827:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760777.774357:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 71024640<>71028736 00000100:00100000:0.0:1789760777.774567:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565449600:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.787810:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984648 gen 3 00000008:00010000:0.0:1789760777.788265:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760777.788274:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.788284:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.788307:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760777.788340:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565449600:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.789139:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17340, 17340] 00000080:00200000:1.0:1789760777.789159:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.789164:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.789181:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71028736 i_blocks 138752 00000020:00200000:1.0:1789760777.789186:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.789245:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.789283:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71028736, count: 4096 00000080:00200000:1.0:1789760777.789291:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71028736, bytes: 4096 00000080:00200000:1.0:1789760777.789304:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.789313:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.789315:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71028736, 71032832) 0 0 00000080:00200000:1.0:1789760777.789320:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.789337:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17341, 17341] 00020000:00200000:1.0:1789760777.789418:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71028736 chunk: [71028736, 71032832) 71032832, 4096 00020000:00200000:1.0:1789760777.789424:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.789445:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17341, 17341] 00000008:00010000:1.0:1789760777.789463:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.789468:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.789480:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71028736, 71032832) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.789494:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71028736=0x43bd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.789567:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.789570:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71032832 00000020:00010000:1.0:1789760777.789579:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.789582:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.789585:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.789588:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.789591:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.789600:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.789607:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.789636:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760777.789799:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 71028736<>71032832 00000100:00100000:0.0:1789760777.790023:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565449728:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.800380:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984649 gen 3 00000008:00010000:0.0:1789760777.800555:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760777.800563:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.800575:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.800689:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760777.800721:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565449728:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.801533:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565449856:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.817163:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17342, 17342] 00000080:00200000:1.0:1789760777.817182:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.817188:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.817286:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71036928 i_blocks 138768 00000020:00200000:0.0:1789760777.817292:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.817304:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.817337:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71036928, count: 4096 00000080:00200000:0.0:1789760777.817345:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71036928, bytes: 4096 00000080:00200000:0.0:1789760777.817359:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.817368:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.817370:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71036928, 71041024) 0 0 00000080:00200000:0.0:1789760777.817375:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.817390:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17343, 17343] 00020000:00200000:0.0:1789760777.817398:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71036928 chunk: [71036928, 71041024) 71041024, 4096 00020000:00200000:0.0:1789760777.817403:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.817422:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17343, 17343] 00000008:00010000:0.0:1789760777.817438:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.817443:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760777.817453:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71036928, 71041024) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.817467:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71036928=0x43bf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.817532:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.817535:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71041024 00000020:00010000:0.0:1789760777.817543:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.817547:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.817550:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.817554:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.817556:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.817564:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.817569:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.817586:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760777.817791:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 71036928<>71041024 00000100:00100000:1.0:1789760777.817864:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565449984:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760777.828441:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984651 gen 3 00000008:00010000:1.0:1789760777.828509:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760777.828528:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760777.828543:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760777.828567:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760777.828600:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565449984:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760777.828683:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17343, 17343] 00000080:00200000:0.0:1789760777.828705:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.828712:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760777.828730:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71041024 i_blocks 138776 00000020:00200000:0.0:1789760777.828734:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760777.828747:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760777.828782:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71041024, count: 4096 00000080:00200000:0.0:1789760777.828792:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71041024, bytes: 4096 00000080:00200000:0.0:1789760777.828803:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760777.828813:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760777.828816:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71041024, 71045120) 0 0 00000080:00200000:0.0:1789760777.828820:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760777.828836:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17344, 17344] 00020000:00200000:0.0:1789760777.828846:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71041024 chunk: [71041024, 71045120) 71045120, 4096 00020000:00200000:0.0:1789760777.828851:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760777.828870:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17344, 17344] 00000008:00010000:0.0:1789760777.828888:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760777.828894:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760777.828905:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71041024, 71045120) append 0 append lockpos 0 00000080:00200000:0.0:1789760777.828919:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71041024=0x43c0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760777.828984:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760777.828987:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71045120 00000020:00010000:0.0:1789760777.828995:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760777.828999:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.829002:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760777.829006:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760777.829008:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760777.829016:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760777.829024:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760777.829042:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760777.829166:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 71041024<>71045120 00000100:00100000:0.0:1789760777.829213:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565450112:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.842699:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984652 gen 3 00000008:00010000:0.0:1789760777.842754:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760777.842762:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.842774:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.842799:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760777.842830:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565450112:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.842949:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17344, 17344] 00000080:00200000:1.0:1789760777.842967:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.842973:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.842989:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71045120 i_blocks 138784 00000020:00200000:1.0:1789760777.842993:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.843005:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.843040:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71045120, count: 4096 00000080:00200000:1.0:1789760777.843047:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71045120, bytes: 4096 00000080:00200000:1.0:1789760777.843059:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.843068:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.843070:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71045120, 71049216) 0 0 00000080:00200000:1.0:1789760777.843074:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.843091:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17345, 17345] 00020000:00200000:1.0:1789760777.843101:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71045120 chunk: [71045120, 71049216) 71049216, 4096 00020000:00200000:1.0:1789760777.843106:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.843128:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17345, 17345] 00000008:00010000:1.0:1789760777.843147:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.843153:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.843166:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71045120, 71049216) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.843181:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71045120=0x43c1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.843243:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.843246:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71049216 00000020:00010000:1.0:1789760777.843257:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.843260:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.843263:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.843267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.843270:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.843278:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.843285:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.843303:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760777.844039:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 71045120<>71049216 00000100:00100000:0.0:1789760777.844144:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760777.844168:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565450240:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.855092:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984653 gen 3 00000008:00010000:0.0:1789760777.855147:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760777.855156:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.855168:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.855182:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760777.855214:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565450240:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.855426:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17345, 17345] 00000080:00200000:1.0:1789760777.855443:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.855449:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.855465:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71049216 i_blocks 138792 00000020:00200000:1.0:1789760777.855469:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.855479:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.855522:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71049216, count: 4096 00000080:00200000:1.0:1789760777.855529:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71049216, bytes: 4096 00000080:00200000:1.0:1789760777.855543:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.855553:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.855555:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71049216, 71053312) 0 0 00000080:00200000:1.0:1789760777.855560:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.855579:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17346, 17346] 00020000:00200000:1.0:1789760777.855587:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71049216 chunk: [71049216, 71053312) 71053312, 4096 00020000:00200000:1.0:1789760777.855592:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.855612:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17346, 17346] 00000008:00010000:1.0:1789760777.855631:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.855637:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.855684:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71049216, 71053312) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.855700:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71049216=0x43c2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.855761:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.855764:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71053312 00000020:00010000:1.0:1789760777.855772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.855776:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.855779:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.855783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.855786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.855795:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.855803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.855820:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760777.855947:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 71049216<>71053312 00000100:00100000:0.0:1789760777.856101:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565450368:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.869548:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984654 gen 3 00000008:00010000:0.0:1789760777.869593:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760777.869601:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.869612:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.869635:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760777.869703:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565450368:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.870007:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17346, 17346] 00000080:00200000:1.0:1789760777.870025:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.870031:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.870045:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71053312 i_blocks 138800 00000020:00200000:1.0:1789760777.870049:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.870059:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.870093:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71053312, count: 4096 00000080:00200000:1.0:1789760777.870101:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71053312, bytes: 4096 00000080:00200000:1.0:1789760777.870111:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.870120:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.870126:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71053312, 71057408) 0 0 00000080:00200000:1.0:1789760777.870130:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.870147:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17347, 17347] 00020000:00200000:1.0:1789760777.870155:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71053312 chunk: [71053312, 71057408) 71057408, 4096 00020000:00200000:1.0:1789760777.870160:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.870184:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17347, 17347] 00000008:00010000:1.0:1789760777.870202:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.870208:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.870219:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71053312, 71057408) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.870232:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71053312=0x43c3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.870290:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.870293:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71057408 00000020:00010000:1.0:1789760777.870301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.870305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.870308:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.870311:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.870313:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.870320:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.870327:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.870342:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760777.870461:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 71053312<>71057408 00000100:00100000:0.0:1789760777.870537:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565450496:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.884951:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984655 gen 3 00000008:00010000:0.0:1789760777.885006:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760777.885015:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.885027:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.885191:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760777.885224:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565450496:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.886191:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17347, 17347] 00000080:00200000:1.0:1789760777.886212:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.886219:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.886750:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71057408 i_blocks 138808 00000020:00200000:1.0:1789760777.886758:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.886774:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.886824:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71057408, count: 4096 00000080:00200000:1.0:1789760777.886833:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71057408, bytes: 4096 00000080:00200000:1.0:1789760777.886848:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.886858:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.886860:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71057408, 71061504) 0 0 00000080:00200000:1.0:1789760777.886865:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.886883:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17348, 17348] 00020000:00200000:1.0:1789760777.886893:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71057408 chunk: [71057408, 71061504) 71061504, 4096 00020000:00200000:1.0:1789760777.886897:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.886921:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17348, 17348] 00000008:00010000:1.0:1789760777.886939:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.886945:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.886957:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71057408, 71061504) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.886971:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71057408=0x43c4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.887048:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.887052:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71061504 00000020:00010000:1.0:1789760777.887061:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.887065:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.887068:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.887084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.887087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.887096:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.887111:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.887142:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760777.887288:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 71057408<>71061504 00000100:00100000:0.0:1789760777.887525:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565450624:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.899431:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984656 gen 3 00000008:00010000:0.0:1789760777.899484:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760777.899494:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.899505:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.899538:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760777.900242:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565450624:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.901060:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17348, 17348] 00000080:00200000:1.0:1789760777.901089:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.901097:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.901116:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71061504 i_blocks 138816 00000020:00200000:1.0:1789760777.901121:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.901133:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.901168:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71061504, count: 4096 00000080:00200000:1.0:1789760777.901176:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71061504, bytes: 4096 00000080:00200000:1.0:1789760777.901186:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.901195:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.901198:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71061504, 71065600) 0 0 00000080:00200000:1.0:1789760777.901203:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.901221:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17349, 17349] 00020000:00200000:1.0:1789760777.901230:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71061504 chunk: [71061504, 71065600) 71065600, 4096 00020000:00200000:1.0:1789760777.901235:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.901256:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17349, 17349] 00000008:00010000:1.0:1789760777.901277:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.901283:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.901295:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71061504, 71065600) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.901310:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71061504=0x43c5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.901376:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.901379:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71065600 00000020:00010000:1.0:1789760777.901388:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.901391:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.901394:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.901397:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.901400:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.901408:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.901414:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.901431:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760777.901580:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 71061504<>71065600 00000100:00100000:0.0:1789760777.901995:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565450752:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.914896:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984657 gen 3 00000008:00010000:0.0:1789760777.914948:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760777.914956:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.914968:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.914990:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760777.915022:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565450752:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760777.915183:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17349, 17349] 00000080:00200000:1.0:1789760777.915200:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760777.915214:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760777.915232:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71065600 i_blocks 138824 00000020:00200000:1.0:1789760777.915236:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760777.915247:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760777.915284:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71065600, count: 4096 00000080:00200000:1.0:1789760777.915291:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71065600, bytes: 4096 00000080:00200000:1.0:1789760777.915301:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760777.915310:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760777.915313:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71065600, 71069696) 0 0 00000080:00200000:1.0:1789760777.915317:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760777.915334:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17350, 17350] 00020000:00200000:1.0:1789760777.915343:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71065600 chunk: [71065600, 71069696) 71069696, 4096 00020000:00200000:1.0:1789760777.915348:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760777.915369:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17350, 17350] 00000008:00010000:1.0:1789760777.915389:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760777.915394:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760777.915406:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71065600, 71069696) append 0 append lockpos 0 00000080:00200000:1.0:1789760777.915420:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71065600=0x43c6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760777.915497:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760777.915500:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71069696 00000020:00010000:1.0:1789760777.915509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760777.915521:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.915524:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760777.915528:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418a1b8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760777.915530:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760777.915538:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760777.915546:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760777.915562:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760777.916327:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 71065600<>71069696 00000100:00100000:0.0:1789760777.916397:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565450880:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760777.923869:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984658 gen 3 00000008:00010000:0.0:1789760777.923928:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760777.923938:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760777.923949:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760777.923977:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760777.924010:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565450880:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760777.944877:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17351, 17351] 00000080:00200000:0.0:1789760777.944906:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760777.944914:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760777.947034:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760777.947059:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565451136:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.957117:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984660 gen 3 00000008:00010000:3.0:1789760777.957173:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760777.957183:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760777.957193:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760777.957220:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760777.957251:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565451136:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.962406:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565451264:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.975151:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984661 gen 3 00000008:00010000:3.0:1789760777.975202:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760777.975211:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760777.975224:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760777.975304:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760777.975335:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565451264:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.979438:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565451392:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.991139:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984662 gen 3 00000008:00010000:3.0:1789760777.991189:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760777.991198:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760777.991209:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760777.991234:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760777.991266:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565451392:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760777.992768:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565451520:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.006424:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984663 gen 3 00000008:00010000:3.0:1789760778.006477:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760778.006575:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.006588:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.006611:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760778.006663:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565451520:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.008292:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760778.008316:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565451648:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.021257:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984664 gen 3 00000008:00010000:3.0:1789760778.021307:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760778.021315:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.021325:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.021348:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760778.021376:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565451648:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760778.032186:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17357, 17357] 00000080:00200000:3.0:1789760778.032211:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760778.032218:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760778.032234:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71098368 i_blocks 138888 00000020:00200000:3.0:1789760778.032238:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760778.032248:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760778.032311:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71098368, count: 4096 00000080:00200000:3.0:1789760778.032318:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71098368, bytes: 4096 00000080:00200000:3.0:1789760778.032329:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760778.032337:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760778.032340:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71098368, 71102464) 0 0 00000080:00200000:3.0:1789760778.032344:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760778.032362:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17358, 17358] 00020000:00200000:3.0:1789760778.032371:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71098368 chunk: [71098368, 71102464) 71102464, 4096 00020000:00200000:3.0:1789760778.032377:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760778.032396:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17358, 17358] 00000008:00010000:3.0:1789760778.032415:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760778.032421:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760778.032433:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71098368, 71102464) append 0 append lockpos 0 00000080:00200000:3.0:1789760778.032689:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71098368=0x43ce000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760778.032750:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760778.032753:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71102464 00000020:00010000:3.0:1789760778.032762:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760778.032765:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.032769:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.032772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760778.032775:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760778.032782:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760778.032789:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760778.032807:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760778.032996:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 71098368<>71102464 00000080:00200000:3.0:1789760778.046945:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17358, 17358] 00000080:00200000:3.0:1789760778.046962:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760778.046969:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760778.046985:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71102464 i_blocks 138896 00000020:00200000:3.0:1789760778.046989:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760778.046999:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760778.047034:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71102464, count: 4096 00000080:00200000:3.0:1789760778.047041:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71102464, bytes: 4096 00000080:00200000:3.0:1789760778.047051:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760778.047061:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760778.047064:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71102464, 71106560) 0 0 00000080:00200000:3.0:1789760778.047068:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760778.047085:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17359, 17359] 00020000:00200000:3.0:1789760778.047094:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71102464 chunk: [71102464, 71106560) 71106560, 4096 00020000:00200000:3.0:1789760778.047099:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760778.047118:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17359, 17359] 00000008:00010000:3.0:1789760778.047136:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760778.047142:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760778.047153:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71102464, 71106560) append 0 append lockpos 0 00000080:00200000:3.0:1789760778.047167:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71102464=0x43cf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760778.047238:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760778.047241:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71106560 00000020:00010000:3.0:1789760778.047250:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760778.047255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.047258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.047261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760778.047264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760778.047273:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760778.047281:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760778.047298:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760778.047418:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 71102464<>71106560 00000080:00200000:3.0:1789760778.057688:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17359, 17359] 00000080:00200000:3.0:1789760778.057706:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760778.057714:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760778.057728:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71106560 i_blocks 138904 00000020:00200000:3.0:1789760778.057731:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760778.057741:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760778.057774:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71106560, count: 4096 00000080:00200000:3.0:1789760778.057783:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71106560, bytes: 4096 00000080:00200000:3.0:1789760778.057792:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760778.057801:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760778.057803:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71106560, 71110656) 0 0 00000080:00200000:3.0:1789760778.057806:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760778.057824:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17360, 17360] 00020000:00200000:3.0:1789760778.057833:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71106560 chunk: [71106560, 71110656) 71110656, 4096 00020000:00200000:3.0:1789760778.057837:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760778.057857:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17360, 17360] 00000008:00010000:3.0:1789760778.057886:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760778.057893:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760778.057905:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71106560, 71110656) append 0 append lockpos 0 00000080:00200000:3.0:1789760778.057919:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71106560=0x43d0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760778.057972:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760778.057975:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71110656 00000020:00010000:3.0:1789760778.057982:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760778.057985:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.057988:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.057991:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760778.057994:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760778.058002:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760778.058009:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:3.0:1789760778.058832:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000080:00200000:3.0:1789760778.070959:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17360, 17360] 00000080:00200000:3.0:1789760778.070977:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760778.071059:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760778.072531:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565452288:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.088980:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17361, 17361] 00000080:00200000:0.0:1789760778.089073:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.089082:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.089100:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71114752 i_blocks 138920 00000020:00200000:0.0:1789760778.089106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.089124:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.089160:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71114752, count: 4096 00000080:00200000:0.0:1789760778.089169:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71114752, bytes: 4096 00000080:00200000:0.0:1789760778.089188:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.089198:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.089200:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71114752, 71118848) 0 0 00000080:00200000:0.0:1789760778.089205:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.089222:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17362, 17362] 00020000:00200000:0.0:1789760778.089231:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71114752 chunk: [71114752, 71118848) 71118848, 4096 00020000:00200000:0.0:1789760778.089236:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.089258:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17362, 17362] 00000008:00010000:0.0:1789760778.089277:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.089283:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.089297:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71114752, 71118848) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.089311:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71114752=0x43d2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.089382:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.089385:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71118848 00000020:00010000:0.0:1789760778.089393:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.089397:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.089400:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.089403:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.089406:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.089415:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.089422:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.089452:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760778.089731:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 71114752<>71118848 00000100:00100000:1.0:1789760778.089826:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565452416:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760778.098752:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984670 gen 3 00000008:00010000:1.0:1789760778.098804:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760778.098823:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760778.098834:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760778.098848:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:1.0:1789760778.099537:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565452416:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.100211:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565452544:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.116392:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17363, 17363] 00000080:00200000:0.0:1789760778.116412:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.116421:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.116438:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71122944 i_blocks 138936 00000020:00200000:0.0:1789760778.116442:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.116610:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.116666:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71122944, count: 4096 00000080:00200000:0.0:1789760778.116676:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71122944, bytes: 4096 00000080:00200000:0.0:1789760778.116691:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.116700:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.116702:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71122944, 71127040) 0 0 00000080:00200000:0.0:1789760778.116707:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.116725:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17364, 17364] 00020000:00200000:0.0:1789760778.116734:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71122944 chunk: [71122944, 71127040) 71127040, 4096 00020000:00200000:0.0:1789760778.116738:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.116762:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17364, 17364] 00000008:00010000:0.0:1789760778.116782:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.116788:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.116801:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71122944, 71127040) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.116816:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71122944=0x43d4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.117558:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.117563:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71127040 00000020:00010000:0.0:1789760778.117573:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.117577:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.117581:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.117584:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.117588:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.117597:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.117606:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.117629:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d300 00000008:00100000:0.0:1789760778.117784:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71122944<>71127040 00000100:00100000:1.0:1789760778.117890:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565452672:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760778.133885:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984672 gen 3 00000008:00010000:1.0:1789760778.134856:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760778.134867:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760778.134879:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760778.134896:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760778.134944:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565452672:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760778.147415:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17365, 17365] 00000080:00200000:3.0:1789760778.147433:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760778.147439:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760778.147456:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71131136 i_blocks 138952 00000020:00200000:3.0:1789760778.147459:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760778.147469:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760778.147635:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71131136, count: 4096 00000080:00200000:3.0:1789760778.147978:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71131136, bytes: 4096 00000080:00200000:3.0:1789760778.147992:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760778.148001:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760778.148004:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71131136, 71135232) 0 0 00000080:00200000:3.0:1789760778.148009:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760778.148160:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17366, 17366] 00020000:00200000:3.0:1789760778.148169:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71131136 chunk: [71131136, 71135232) 71135232, 4096 00020000:00200000:3.0:1789760778.148174:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760778.148196:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17366, 17366] 00000008:00010000:3.0:1789760778.148214:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760778.148219:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760778.148231:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71131136, 71135232) append 0 append lockpos 0 00000080:00200000:3.0:1789760778.148245:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71131136=0x43d6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760778.148311:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760778.148314:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71135232 00000020:00010000:3.0:1789760778.148322:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760778.148325:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.148328:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.148331:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760778.148334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760778.148344:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760778.148350:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760778.148372:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760778.148669:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 71131136<>71135232 00000080:00200000:0.0:1789760778.175269:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17366, 17366] 00000080:00200000:0.0:1789760778.175286:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.175294:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.175383:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71135232 i_blocks 138960 00000020:00200000:0.0:1789760778.175388:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.175398:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.175433:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71135232, count: 4096 00000080:00200000:0.0:1789760778.175440:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71135232, bytes: 4096 00000080:00200000:0.0:1789760778.175779:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.175790:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.175792:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71135232, 71139328) 0 0 00000080:00200000:0.0:1789760778.175796:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.175813:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17367, 17367] 00020000:00200000:0.0:1789760778.175822:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71135232 chunk: [71135232, 71139328) 71139328, 4096 00020000:00200000:0.0:1789760778.175827:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.175849:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17367, 17367] 00000008:00010000:0.0:1789760778.175943:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.175950:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.175963:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71135232, 71139328) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.175977:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71135232=0x43d7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.176038:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.176110:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71139328 00000020:00010000:0.0:1789760778.176119:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.176123:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.176126:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.176129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.176132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.176141:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.176148:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.176173:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.176378:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71135232<>71139328 00000100:00100000:1.0:1789760778.176670:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565453056:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.188043:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984675 gen 3 00000008:00010000:0.0:1789760778.188096:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.188106:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.188117:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.188146:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.188177:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565453056:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.189328:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565453184:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.203258:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984676 gen 3 00000008:00010000:3.0:1789760778.203308:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760778.203316:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.203328:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.203346:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760778.203377:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565453184:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.204383:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565453312:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.217593:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984677 gen 3 00000008:00010000:3.0:1789760778.217769:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760778.217779:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.217790:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.217867:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760778.217898:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565453312:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760778.232497:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17370, 17370] 00000080:00200000:3.0:1789760778.232533:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760778.232541:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760778.232560:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71151616 i_blocks 138992 00000020:00200000:3.0:1789760778.232566:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760778.232583:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760778.232710:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71151616, count: 4096 00000080:00200000:3.0:1789760778.232719:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71151616, bytes: 4096 00000080:00200000:3.0:1789760778.232735:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760778.232746:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760778.232749:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71151616, 71155712) 0 0 00000080:00200000:3.0:1789760778.232754:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760778.232770:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17371, 17371] 00020000:00200000:3.0:1789760778.232779:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71151616 chunk: [71151616, 71155712) 71155712, 4096 00020000:00200000:3.0:1789760778.232784:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760778.232806:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17371, 17371] 00000008:00010000:3.0:1789760778.232824:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760778.232831:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760778.232845:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71151616, 71155712) append 0 append lockpos 0 00000080:00200000:3.0:1789760778.232861:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71151616=0x43db000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760778.233006:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760778.233009:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71155712 00000020:00010000:3.0:1789760778.233019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760778.233023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.233026:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567eeb0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.233029:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760778.233032:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567eeb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760778.233041:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760778.233049:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760778.233079:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760778.233229:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 71151616<>71155712 00000100:00100000:3.0:1789760778.233292:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760778.233316:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565453568:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.245948:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984679 gen 3 00000008:00010000:3.0:1789760778.246001:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:3.0:1789760778.246009:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.246022:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.246048:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:3.0:1789760778.246079:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565453568:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.246237:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17371, 17371] 00000080:00200000:0.0:1789760778.246255:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.246261:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.246276:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71155712 i_blocks 139000 00000020:00200000:0.0:1789760778.246280:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.246290:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.246324:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71155712, count: 4096 00000080:00200000:0.0:1789760778.246331:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71155712, bytes: 4096 00000080:00200000:0.0:1789760778.246342:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.246351:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.246354:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71155712, 71159808) 0 0 00000080:00200000:0.0:1789760778.246358:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.246376:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17372, 17372] 00020000:00200000:0.0:1789760778.246385:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71155712 chunk: [71155712, 71159808) 71159808, 4096 00020000:00200000:0.0:1789760778.246390:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.246411:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17372, 17372] 00000008:00010000:0.0:1789760778.246429:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.246435:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.246457:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71155712, 71159808) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.246472:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71155712=0x43dc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.246542:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.246545:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71159808 00000020:00010000:0.0:1789760778.246554:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.246559:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.246562:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.246566:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.246570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.246579:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.246586:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.246603:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.246789:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71155712<>71159808 00000100:00100000:0.0:1789760778.246826:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760778.246845:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565453696:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.259936:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984680 gen 3 00000008:00010000:0.0:1789760778.259990:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.259998:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.260009:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.260133:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.260165:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565453696:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760778.260251:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17372, 17372] 00000080:00200000:3.0:1789760778.260268:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760778.260275:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760778.260290:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71159808 i_blocks 139008 00000020:00200000:3.0:1789760778.260294:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760778.260306:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760778.260340:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71159808, count: 4096 00000080:00200000:3.0:1789760778.260347:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71159808, bytes: 4096 00000080:00200000:3.0:1789760778.260357:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760778.260366:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760778.260368:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71159808, 71163904) 0 0 00000080:00200000:3.0:1789760778.260373:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760778.260524:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17373, 17373] 00020000:00200000:3.0:1789760778.260535:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71159808 chunk: [71159808, 71163904) 71163904, 4096 00020000:00200000:3.0:1789760778.260540:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760778.260569:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17373, 17373] 00000008:00010000:3.0:1789760778.260588:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760778.260594:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567eeb0, osc lock ffff93f8457e7a20, flags 0x0 00000080:00200000:3.0:1789760778.260605:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71159808, 71163904) append 0 append lockpos 0 00000080:00200000:3.0:1789760778.260620:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71159808=0x43dd000, pages 1 (max 131072), lockless, parallel 00000100:00100000:3.0:1789760778.261228:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565453824:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.273845:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17373, 17373] 00000080:00200000:0.0:1789760778.273934:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.273941:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.273957:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71163904 i_blocks 139016 00000020:00200000:0.0:1789760778.273961:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.273973:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.274009:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71163904, count: 4096 00000080:00200000:0.0:1789760778.274017:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71163904, bytes: 4096 00000080:00200000:0.0:1789760778.274029:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.274037:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.274040:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71163904, 71168000) 0 0 00000080:00200000:0.0:1789760778.274044:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.274062:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17374, 17374] 00020000:00200000:0.0:1789760778.274071:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71163904 chunk: [71163904, 71168000) 71168000, 4096 00020000:00200000:0.0:1789760778.274076:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.274095:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17374, 17374] 00000008:00010000:0.0:1789760778.274113:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.274120:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.274133:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71163904, 71168000) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.274146:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71163904=0x43de000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.274210:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.274213:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71168000 00000020:00010000:0.0:1789760778.274221:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.274224:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.274228:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.274232:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.274235:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.274244:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.274251:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.274271:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.274401:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71163904<>71168000 00000100:00100000:1.0:1789760778.274864:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565453952:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.288006:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984682 gen 3 00000008:00010000:0.0:1789760778.288070:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.288079:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.288089:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.288198:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.288229:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565453952:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.304721:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17375, 17375] 00000080:00200000:0.0:1789760778.304748:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.304756:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.304774:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71172096 i_blocks 139032 00000020:00200000:0.0:1789760778.304780:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.305041:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.305080:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71172096, count: 4096 00000080:00200000:0.0:1789760778.305090:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71172096, bytes: 4096 00000080:00200000:0.0:1789760778.305106:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.305116:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.305119:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71172096, 71176192) 0 0 00000080:00200000:0.0:1789760778.305125:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.305144:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17376, 17376] 00020000:00200000:0.0:1789760778.305154:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71172096 chunk: [71172096, 71176192) 71176192, 4096 00020000:00200000:0.0:1789760778.305159:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.305184:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17376, 17376] 00000008:00010000:0.0:1789760778.305201:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.305207:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.305219:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71172096, 71176192) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.305243:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71172096=0x43e0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.305329:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.305339:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71176192 00000020:00010000:0.0:1789760778.305347:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.305350:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.305353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.305357:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.305360:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.305369:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.305376:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.305400:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.305568:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71172096<>71176192 00000100:00100000:0.0:1789760778.305635:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760778.305733:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565454208:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.320742:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984684 gen 3 00000008:00010000:0.0:1789760778.320793:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.320801:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.320811:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.320834:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.320863:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565454208:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.332018:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17377, 17377] 00000080:00200000:0.0:1789760778.332036:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.332043:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.332060:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71180288 i_blocks 139048 00000020:00200000:0.0:1789760778.332064:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.332076:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.332110:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71180288, count: 4096 00000080:00200000:0.0:1789760778.332118:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71180288, bytes: 4096 00000080:00200000:0.0:1789760778.332129:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.332139:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.332142:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71180288, 71184384) 0 0 00000080:00200000:0.0:1789760778.332147:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.332406:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17378, 17378] 00020000:00200000:0.0:1789760778.332415:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71180288 chunk: [71180288, 71184384) 71184384, 4096 00020000:00200000:0.0:1789760778.332419:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.332440:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17378, 17378] 00000008:00010000:0.0:1789760778.332458:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.332464:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.332476:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71180288, 71184384) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.332490:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71180288=0x43e2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.332601:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.332604:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71184384 00000020:00010000:0.0:1789760778.332721:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.332726:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.332729:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.332733:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.332736:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.332744:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.332751:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.332773:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.332892:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71180288<>71184384 00000100:00100000:1.0:1789760778.333025:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565454464:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.346507:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984686 gen 3 00000008:00010000:0.0:1789760778.346562:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.346570:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.346580:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.346857:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.346889:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565454464:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.363895:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17379, 17379] 00000080:00200000:0.0:1789760778.363924:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.363931:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.363953:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71188480 i_blocks 139064 00000020:00200000:0.0:1789760778.363959:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.363977:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.364016:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71188480, count: 4096 00000080:00200000:0.0:1789760778.364025:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71188480, bytes: 4096 00000080:00200000:0.0:1789760778.364042:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.364056:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.364059:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71188480, 71192576) 0 0 00000080:00200000:0.0:1789760778.364063:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.364082:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17380, 17380] 00020000:00200000:0.0:1789760778.364092:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71188480 chunk: [71188480, 71192576) 71192576, 4096 00020000:00200000:0.0:1789760778.364097:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.364123:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17380, 17380] 00000008:00010000:0.0:1789760778.364144:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.364163:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.364178:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71188480, 71192576) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.364195:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71188480=0x43e4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.364272:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.364276:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71192576 00000020:00010000:0.0:1789760778.364284:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.364289:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.364292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.364295:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.364298:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.364307:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.364315:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.364347:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.364520:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71188480<>71192576 00000100:00100000:0.0:1789760778.364577:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760778.365235:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565454720:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.380766:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984688 gen 3 00000008:00010000:0.0:1789760778.380817:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.380825:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.380836:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.380848:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.380878:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565454720:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.380988:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17380, 17380] 00000080:00200000:1.0:1789760778.381005:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.381010:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.381026:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71192576 i_blocks 139072 00000020:00200000:1.0:1789760778.381030:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.381042:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.381086:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71192576, count: 4096 00000080:00200000:1.0:1789760778.381094:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71192576, bytes: 4096 00000080:00200000:1.0:1789760778.381105:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.381114:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.381117:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71192576, 71196672) 0 0 00000080:00200000:1.0:1789760778.381121:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.381138:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17381, 17381] 00020000:00200000:1.0:1789760778.381146:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71192576 chunk: [71192576, 71196672) 71196672, 4096 00020000:00200000:1.0:1789760778.381151:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.381171:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17381, 17381] 00000008:00010000:1.0:1789760778.381188:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.381194:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.381207:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71192576, 71196672) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.381222:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71192576=0x43e5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.381282:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.381285:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71196672 00000020:00010000:1.0:1789760778.381294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.381298:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.381301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.381305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.381308:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.381315:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.381323:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.381339:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760778.381476:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 71192576<>71196672 00000100:00100000:1.0:1789760778.381535:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760778.381559:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565454848:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.394124:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984689 gen 3 00000008:00010000:0.0:1789760778.394173:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760778.394182:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.394194:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.394282:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760778.394315:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565454848:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.395181:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565454976:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.413750:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984690 gen 3 00000008:00010000:3.0:1789760778.413801:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760778.413809:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.413820:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.413841:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760778.413873:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565454976:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.415122:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17382, 17382] 00000080:00200000:1.0:1789760778.415145:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.415152:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.415168:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71200768 i_blocks 139088 00000020:00200000:1.0:1789760778.415173:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.415186:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.415221:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71200768, count: 4096 00000080:00200000:1.0:1789760778.415229:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71200768, bytes: 4096 00000080:00200000:1.0:1789760778.415243:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.415252:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.415255:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71200768, 71204864) 0 0 00000080:00200000:1.0:1789760778.415260:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.415349:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17383, 17383] 00020000:00200000:1.0:1789760778.415359:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71200768 chunk: [71200768, 71204864) 71204864, 4096 00020000:00200000:1.0:1789760778.415364:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.415390:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17383, 17383] 00000008:00010000:1.0:1789760778.415409:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.415416:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.415641:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71200768, 71204864) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.415697:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71200768=0x43e7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.415769:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.415772:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71204864 00000020:00010000:1.0:1789760778.415781:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.415784:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.415787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.415790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.415793:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.415801:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.415809:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.415830:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760778.415971:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 71200768<>71204864 00000100:00100000:0.0:1789760778.416062:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565455104:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.429696:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984691 gen 3 00000008:00010000:0.0:1789760778.429762:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760778.429771:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.429782:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.429804:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760778.429835:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565455104:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.429867:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17383, 17383] 00000080:00200000:1.0:1789760778.429897:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.429904:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.429928:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71204864 i_blocks 139096 00000020:00200000:1.0:1789760778.429933:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.429943:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.429977:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71204864, count: 4096 00000080:00200000:1.0:1789760778.429985:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71204864, bytes: 4096 00000080:00200000:1.0:1789760778.429995:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.430004:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.430007:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71204864, 71208960) 0 0 00000080:00200000:1.0:1789760778.430011:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.430030:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17384, 17384] 00020000:00200000:1.0:1789760778.430040:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71204864 chunk: [71204864, 71208960) 71208960, 4096 00020000:00200000:1.0:1789760778.430045:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.430065:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17384, 17384] 00000008:00010000:1.0:1789760778.430083:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.430089:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.430100:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71204864, 71208960) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.430113:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71204864=0x43e8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.430178:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.430181:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71208960 00000020:00010000:1.0:1789760778.430188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.430192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.430195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.430198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.430201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.430210:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.430216:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.430233:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760778.430460:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 71204864<>71208960 00000100:00100000:0.0:1789760778.430563:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565455232:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.441298:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984692 gen 3 00000008:00010000:0.0:1789760778.441349:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760778.441358:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.441368:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.441428:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760778.441594:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565455232:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.441765:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17384, 17384] 00000080:00200000:1.0:1789760778.441781:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.441787:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.441803:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71208960 i_blocks 139104 00000020:00200000:1.0:1789760778.441807:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.441817:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.441851:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71208960, count: 4096 00000080:00200000:1.0:1789760778.441999:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71208960, bytes: 4096 00000080:00200000:1.0:1789760778.442012:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.442021:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.442024:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71208960, 71213056) 0 0 00000080:00200000:1.0:1789760778.442029:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.442046:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17385, 17385] 00020000:00200000:1.0:1789760778.442055:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71208960 chunk: [71208960, 71213056) 71213056, 4096 00020000:00200000:1.0:1789760778.442060:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.442082:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17385, 17385] 00000008:00010000:1.0:1789760778.442100:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.442107:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.442118:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71208960, 71213056) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.442132:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71208960=0x43e9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.442195:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.442198:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71213056 00000020:00010000:1.0:1789760778.442207:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.442211:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.442214:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.442218:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.442221:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.442228:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.442234:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.442253:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760778.442383:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 71208960<>71213056 00000100:00100000:0.0:1789760778.442464:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565455360:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.456164:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984693 gen 3 00000008:00010000:0.0:1789760778.456213:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760778.456222:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.456232:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.456287:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760778.456313:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565455360:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.457401:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17385, 17385] 00000080:00200000:1.0:1789760778.457421:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.457428:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.457445:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71213056 i_blocks 139112 00000020:00200000:1.0:1789760778.457449:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.457462:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.457496:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71213056, count: 4096 00000080:00200000:1.0:1789760778.457504:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71213056, bytes: 4096 00000080:00200000:1.0:1789760778.457529:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.457538:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.457541:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71213056, 71217152) 0 0 00000080:00200000:1.0:1789760778.457545:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.457562:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17386, 17386] 00020000:00200000:1.0:1789760778.457571:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71213056 chunk: [71213056, 71217152) 71217152, 4096 00020000:00200000:1.0:1789760778.457576:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.457596:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17386, 17386] 00000008:00010000:1.0:1789760778.457612:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.457619:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.457631:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71213056, 71217152) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.457662:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71213056=0x43ea000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.457807:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.457811:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71217152 00000020:00010000:1.0:1789760778.457821:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.457824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.457827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.457831:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.457833:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.457841:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.457849:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.457875:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760778.459533:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 71213056<>71217152 00000100:00100000:0.0:1789760778.459630:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565455488:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.467063:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984694 gen 3 00000008:00010000:0.0:1789760778.467115:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760778.467124:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.467134:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.467152:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760778.467182:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565455488:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.467368:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17386, 17386] 00000080:00200000:1.0:1789760778.467394:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.467401:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.467416:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71217152 i_blocks 139120 00000020:00200000:1.0:1789760778.467420:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.467430:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.467465:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71217152, count: 4096 00000080:00200000:1.0:1789760778.467473:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71217152, bytes: 4096 00000080:00200000:1.0:1789760778.467483:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.467638:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.467641:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71217152, 71221248) 0 0 00000080:00200000:1.0:1789760778.467692:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.467711:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17387, 17387] 00020000:00200000:1.0:1789760778.467721:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71217152 chunk: [71217152, 71221248) 71221248, 4096 00020000:00200000:1.0:1789760778.467726:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.467748:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17387, 17387] 00000008:00010000:1.0:1789760778.467768:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.467774:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.467786:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71217152, 71221248) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.467806:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71217152=0x43eb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.468470:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.468475:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71221248 00000020:00010000:1.0:1789760778.468498:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.468502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.468506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.468519:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.468522:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.468532:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.468540:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.468559:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760778.468716:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 71217152<>71221248 00000100:00100000:0.0:1789760778.468923:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760778.468948:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565455616:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.480394:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984695 gen 3 00000008:00010000:0.0:1789760778.480444:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760778.480454:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.480465:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.480485:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760778.480527:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565455616:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.480634:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17387, 17387] 00000080:00200000:1.0:1789760778.480685:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.480692:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.480706:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71221248 i_blocks 139128 00000020:00200000:1.0:1789760778.480709:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.480718:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.480748:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71221248, count: 4096 00000080:00200000:1.0:1789760778.480755:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71221248, bytes: 4096 00000080:00200000:1.0:1789760778.480764:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.480773:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.480775:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71221248, 71225344) 0 0 00000080:00200000:1.0:1789760778.480780:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.480795:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17388, 17388] 00020000:00200000:1.0:1789760778.480804:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71221248 chunk: [71221248, 71225344) 71225344, 4096 00020000:00200000:1.0:1789760778.480808:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.480828:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17388, 17388] 00000008:00010000:1.0:1789760778.480845:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.480851:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.480863:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71221248, 71225344) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.480882:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71221248=0x43ec000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.480939:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.480942:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71225344 00000020:00010000:1.0:1789760778.480951:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.480954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.480957:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.480961:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.480963:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.480970:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.480982:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.481009:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760778.481133:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 71221248<>71225344 00000100:00100000:0.0:1789760778.481309:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565455744:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.491372:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984696 gen 3 00000008:00010000:0.0:1789760778.491417:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760778.491425:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.491435:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.491455:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760778.491482:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565455744:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.492545:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17388, 17388] 00000080:00200000:1.0:1789760778.492565:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.492570:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.492586:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71225344 i_blocks 139136 00000020:00200000:1.0:1789760778.492590:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.492602:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.492634:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71225344, count: 4096 00000080:00200000:1.0:1789760778.492641:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71225344, bytes: 4096 00000080:00200000:1.0:1789760778.492694:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.492703:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.492705:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71225344, 71229440) 0 0 00000080:00200000:1.0:1789760778.492709:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.492724:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17389, 17389] 00020000:00200000:1.0:1789760778.492732:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71225344 chunk: [71225344, 71229440) 71229440, 4096 00020000:00200000:1.0:1789760778.492736:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.492756:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17389, 17389] 00000008:00010000:1.0:1789760778.492774:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.492780:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.492792:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71225344, 71229440) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.492805:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71225344=0x43ed000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.492862:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.492865:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71229440 00000020:00010000:1.0:1789760778.492873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.492876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.492879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.492883:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.492886:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.492893:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.492899:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.492916:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760778.493029:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 71225344<>71229440 00000100:00100000:0.0:1789760778.493200:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760778.493223:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565455872:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.506359:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984697 gen 3 00000008:00010000:0.0:1789760778.506413:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760778.506422:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.506434:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.506503:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760778.506541:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565455872:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.506690:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17389, 17389] 00000080:00200000:1.0:1789760778.506707:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.506713:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.506728:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71229440 i_blocks 139144 00000020:00200000:1.0:1789760778.506732:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.506742:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.506777:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71229440, count: 4096 00000080:00200000:1.0:1789760778.506785:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71229440, bytes: 4096 00000080:00200000:1.0:1789760778.506795:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.506805:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.506807:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71229440, 71233536) 0 0 00000080:00200000:1.0:1789760778.506812:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.506835:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17390, 17390] 00020000:00200000:1.0:1789760778.506844:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71229440 chunk: [71229440, 71233536) 71233536, 4096 00020000:00200000:1.0:1789760778.506849:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.506870:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17390, 17390] 00000008:00010000:1.0:1789760778.506889:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.506895:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.506974:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71229440, 71233536) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.506990:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71229440=0x43ee000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.507055:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.507059:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71233536 00000020:00010000:1.0:1789760778.507067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.507070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.507073:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.507077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.507080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.507088:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.507094:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.507114:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760778.510219:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 71229440<>71233536 00000100:00100000:0.0:1789760778.510305:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565456000:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.523331:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984698 gen 3 00000008:00010000:0.0:1789760778.523377:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760778.523385:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.523395:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.523426:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760778.523463:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565456000:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.525703:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565456128:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.539133:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984699 gen 3 00000008:00010000:3.0:1789760778.539188:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760778.539197:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.539207:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.539233:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760778.539265:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565456128:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.542195:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565456256:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.552432:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984700 gen 3 00000008:00010000:3.0:1789760778.552573:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760778.552582:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.552592:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.552617:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760778.552691:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565456256:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.554819:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760778.554850:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565456384:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.568193:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984701 gen 3 00000008:00010000:3.0:1789760778.568243:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760778.568252:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.568264:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.568342:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760778.568371:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565456384:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.571200:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565456512:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.584272:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984702 gen 3 00000008:00010000:3.0:1789760778.584324:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760778.584333:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.584343:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.584368:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760778.584398:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565456512:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.585680:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565456640:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.600840:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984703 gen 3 00000008:00010000:3.0:1789760778.600893:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760778.600901:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.600912:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.600938:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760778.600967:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565456640:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.602287:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760778.602311:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565456768:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.616101:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984704 gen 3 00000008:00010000:3.0:1789760778.616154:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760778.616162:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.616174:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.616198:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760778.616826:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565456768:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760778.616883:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:3.0:1789760778.617020:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 71258112<>71262208 00000100:00100000:3.0:1789760778.617077:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565456896:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.629911:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984705 gen 3 00000008:00010000:3.0:1789760778.629964:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760778.629974:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.629985:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.630017:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760778.630049:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565456896:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.632072:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565457024:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.640788:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984706 gen 3 00000008:00010000:3.0:1789760778.640841:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760778.640849:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.640873:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.640895:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760778.640933:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565457024:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.642340:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760778.642361:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565457152:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.651124:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984707 gen 3 00000008:00010000:3.0:1789760778.651173:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760778.651182:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.651197:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.651221:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760778.651251:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565457152:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.655952:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760778.655979:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565457280:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.670061:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17400, 17400] 00000080:00200000:0.0:1789760778.670081:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.670087:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.670102:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71274496 i_blocks 139232 00000020:00200000:0.0:1789760778.670106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.670117:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.670151:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71274496, count: 4096 00000080:00200000:0.0:1789760778.670158:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71274496, bytes: 4096 00000080:00200000:0.0:1789760778.670170:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.670179:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.670181:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71274496, 71278592) 0 0 00000080:00200000:0.0:1789760778.670185:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.670201:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17401, 17401] 00020000:00200000:0.0:1789760778.670210:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71274496 chunk: [71274496, 71278592) 71278592, 4096 00020000:00200000:0.0:1789760778.670214:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.670249:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17401, 17401] 00000008:00010000:0.0:1789760778.670269:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.670275:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.670287:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71274496, 71278592) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.670300:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71274496=0x43f9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.670365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.670368:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71278592 00000020:00010000:0.0:1789760778.670377:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.670381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.670384:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.670387:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.670390:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.670400:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.670407:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.670425:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.670562:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71274496<>71278592 00000100:00100000:1.0:1789760778.670638:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565457408:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.682877:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984709 gen 3 00000008:00010000:0.0:1789760778.682922:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.682930:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.682940:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.682966:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.683000:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565457408:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760778.706335:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17402, 17402] 00000080:00200000:3.0:1789760778.706351:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760778.706357:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760778.706372:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71282688 i_blocks 139248 00000020:00200000:3.0:1789760778.706376:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760778.706399:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760778.706433:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71282688, count: 4096 00000080:00200000:3.0:1789760778.706440:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71282688, bytes: 4096 00000080:00200000:3.0:1789760778.706460:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760778.706469:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760778.706472:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71282688, 71286784) 0 0 00000080:00200000:3.0:1789760778.706476:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760778.706494:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17403, 17403] 00020000:00200000:3.0:1789760778.706503:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71282688 chunk: [71282688, 71286784) 71286784, 4096 00020000:00200000:3.0:1789760778.706516:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760778.706536:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17403, 17403] 00000008:00010000:3.0:1789760778.706562:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760778.706567:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760778.706580:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71282688, 71286784) append 0 append lockpos 0 00000080:00200000:3.0:1789760778.706597:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71282688=0x43fb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760778.706890:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760778.706893:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71286784 00000020:00010000:3.0:1789760778.706902:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760778.706905:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.706908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760778.706911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760778.706915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760778.706924:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760778.706931:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760778.706951:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760778.707084:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 71282688<>71286784 00000100:00100000:3.0:1789760778.707133:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760778.707154:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565457664:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.723255:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984711 gen 3 00000008:00010000:3.0:1789760778.723304:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:3.0:1789760778.723313:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.723325:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.723347:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:3.0:1789760778.723377:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565457664:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.723832:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17403, 17403] 00000080:00200000:0.0:1789760778.723859:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.723867:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.723882:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71286784 i_blocks 139256 00000020:00200000:0.0:1789760778.723886:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.723897:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.723931:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71286784, count: 4096 00000080:00200000:0.0:1789760778.723938:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71286784, bytes: 4096 00000080:00200000:0.0:1789760778.723948:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.723956:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.723958:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71286784, 71290880) 0 0 00000080:00200000:0.0:1789760778.723962:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.723978:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17404, 17404] 00020000:00200000:0.0:1789760778.723987:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71286784 chunk: [71286784, 71290880) 71290880, 4096 00020000:00200000:0.0:1789760778.723991:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.724010:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17404, 17404] 00000008:00010000:0.0:1789760778.724149:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.724157:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.724170:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71286784, 71290880) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.724183:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71286784=0x43fc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.724251:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.724255:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71290880 00000020:00010000:0.0:1789760778.724263:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.724267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.724270:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.724274:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.724277:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.724285:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.724292:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.724310:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.724433:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71286784<>71290880 00000100:00100000:0.0:1789760778.725207:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565457792:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.737122:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984712 gen 3 00000008:00010000:0.0:1789760778.737169:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.737178:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.737189:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.737249:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.737281:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565457792:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.737703:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17404, 17404] 00000080:00200000:1.0:1789760778.737722:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.737728:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.737745:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71290880 i_blocks 139264 00000020:00200000:1.0:1789760778.737749:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.737761:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.738063:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71290880, count: 4096 00000080:00200000:1.0:1789760778.738073:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71290880, bytes: 4096 00000080:00200000:1.0:1789760778.738085:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.738095:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.738097:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71290880, 71294976) 0 0 00000080:00200000:1.0:1789760778.738102:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.738122:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17405, 17405] 00020000:00200000:1.0:1789760778.738131:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71290880 chunk: [71290880, 71294976) 71294976, 4096 00020000:00200000:1.0:1789760778.738136:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.738158:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17405, 17405] 00000008:00010000:1.0:1789760778.738179:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.738185:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.738199:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71290880, 71294976) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.738215:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71290880=0x43fd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.738281:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.738284:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71294976 00000020:00010000:1.0:1789760778.738293:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.738296:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.738299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.738303:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.738306:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.738314:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.738321:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.738340:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760778.738597:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 71290880<>71294976 00000100:00100000:0.0:1789760778.738682:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565457920:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.746341:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984713 gen 3 00000008:00010000:0.0:1789760778.746635:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760778.746696:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.746709:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.746722:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760778.746751:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565457920:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.746929:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17405, 17405] 00000080:00200000:1.0:1789760778.746947:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.746953:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.746968:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71294976 i_blocks 139272 00000020:00200000:1.0:1789760778.746972:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.747051:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.747085:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71294976, count: 4096 00000080:00200000:1.0:1789760778.747093:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71294976, bytes: 4096 00000080:00200000:1.0:1789760778.747104:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.747114:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.747116:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71294976, 71299072) 0 0 00000080:00200000:1.0:1789760778.747121:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.747139:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17406, 17406] 00020000:00200000:1.0:1789760778.747149:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71294976 chunk: [71294976, 71299072) 71299072, 4096 00020000:00200000:1.0:1789760778.747154:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.747174:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17406, 17406] 00000008:00010000:1.0:1789760778.747192:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.747199:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.747211:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71294976, 71299072) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.747223:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71294976=0x43fe000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.747398:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.747401:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71299072 00000020:00010000:1.0:1789760778.747410:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.747414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.747418:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.747421:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.747424:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.747432:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.747438:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.747455:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760778.747682:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 71294976<>71299072 00000100:00100000:0.0:1789760778.747798:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458048:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.762012:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984714 gen 3 00000008:00010000:0.0:1789760778.762068:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760778.762077:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.762087:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.762108:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760778.762139:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458048:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.762727:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17406, 17406] 00000080:00200000:1.0:1789760778.762744:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.762751:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.762767:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71299072 i_blocks 139280 00000020:00200000:1.0:1789760778.762771:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.762794:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.762844:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71299072, count: 4096 00000080:00200000:1.0:1789760778.762852:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71299072, bytes: 4096 00000080:00200000:1.0:1789760778.762864:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.762877:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.762880:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71299072, 71303168) 0 0 00000080:00200000:1.0:1789760778.763229:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.763249:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17407, 17407] 00020000:00200000:1.0:1789760778.763259:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71299072 chunk: [71299072, 71303168) 71303168, 4096 00020000:00200000:1.0:1789760778.763264:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.763286:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17407, 17407] 00000008:00010000:1.0:1789760778.763305:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.763313:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.763327:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71299072, 71303168) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.763342:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71299072=0x43ff000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.763414:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.763417:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71303168 00000020:00010000:1.0:1789760778.763426:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.763430:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.763433:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.763436:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.763439:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.763461:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.763470:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.763496:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760778.763719:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 71299072<>71303168 00000100:00100000:0.0:1789760778.763916:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760778.763948:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458176:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.777868:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984715 gen 3 00000008:00010000:0.0:1789760778.777920:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760778.777928:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.777939:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.778061:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760778.778094:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458176:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.779440:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17407, 17407] 00000080:00200000:1.0:1789760778.779462:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.779469:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.779486:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71303168 i_blocks 139288 00000020:00200000:1.0:1789760778.779490:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.779502:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00010000:00010000:3.0:1789760778.781663:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1789760778.783619:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71303168, count: 4096 00000080:00200000:1.0:1789760778.783638:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71303168, bytes: 4096 00000080:00200000:1.0:1789760778.783670:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.783681:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.783684:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71303168, 71307264) 0 0 00000080:00200000:1.0:1789760778.783690:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.783711:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17408, 17408] 00020000:00200000:1.0:1789760778.783723:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71303168 chunk: [71303168, 71307264) 71307264, 4096 00020000:00200000:1.0:1789760778.783728:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.783755:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17408, 17408] 00000008:00010000:1.0:1789760778.783774:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.783781:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.783794:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71303168, 71307264) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.783809:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71303168=0x4400000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.783890:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.783893:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71307264 00000020:00010000:1.0:1789760778.783902:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.783905:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.783908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.783912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.783916:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.783929:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.783936:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.783985:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760778.784121:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 71303168<>71307264 00000100:00100000:0.0:1789760778.784283:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458304:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.796834:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984716 gen 3 00000008:00010000:0.0:1789760778.796886:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760778.796895:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.796906:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.796929:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760778.796959:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458304:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.797321:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17408, 17408] 00000080:00200000:1.0:1789760778.797339:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.797345:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.797360:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71307264 i_blocks 139296 00000020:00200000:1.0:1789760778.797363:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.797374:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.797408:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71307264, count: 4096 00000080:00200000:1.0:1789760778.797416:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71307264, bytes: 4096 00000080:00200000:1.0:1789760778.797427:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.797435:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.797438:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71307264, 71311360) 0 0 00000080:00200000:1.0:1789760778.797442:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.797458:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17409, 17409] 00020000:00200000:1.0:1789760778.797467:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71307264 chunk: [71307264, 71311360) 71311360, 4096 00020000:00200000:1.0:1789760778.797472:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.797493:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17409, 17409] 00000008:00010000:1.0:1789760778.797519:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.797527:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.797539:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71307264, 71311360) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.797554:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71307264=0x4401000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.797614:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.797617:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71311360 00000020:00010000:1.0:1789760778.797625:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.797629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.797632:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.797635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.797638:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.797669:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.797677:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.797694:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760778.797818:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 71307264<>71311360 00000100:00100000:0.0:1789760778.797997:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760778.798016:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458432:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.808740:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984717 gen 3 00000008:00010000:0.0:1789760778.808793:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760778.808801:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.808812:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.808836:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760778.808865:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458432:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.808983:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17409, 17409] 00000080:00200000:1.0:1789760778.808999:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.809005:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.809020:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71311360 i_blocks 139304 00000020:00200000:1.0:1789760778.809023:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.809032:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.809071:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71311360, count: 4096 00000080:00200000:1.0:1789760778.809078:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71311360, bytes: 4096 00000080:00200000:1.0:1789760778.809088:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.809096:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.809099:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71311360, 71315456) 0 0 00000080:00200000:1.0:1789760778.809103:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.809120:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17410, 17410] 00020000:00200000:1.0:1789760778.809130:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71311360 chunk: [71311360, 71315456) 71315456, 4096 00020000:00200000:1.0:1789760778.809134:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.809154:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17410, 17410] 00000008:00010000:1.0:1789760778.809171:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.809177:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.809188:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71311360, 71315456) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.809202:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71311360=0x4402000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.809267:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.809270:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71315456 00000020:00010000:1.0:1789760778.809279:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.809283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.809285:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.809288:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.809292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.809299:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.809316:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.809332:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760778.809468:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 71311360<>71315456 00000100:00100000:0.0:1789760778.809698:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458560:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.817218:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984718 gen 3 00000008:00010000:0.0:1789760778.817352:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760778.817360:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.817371:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.817596:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760778.817638:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458560:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.817983:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17410, 17410] 00000080:00200000:1.0:1789760778.818000:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.818006:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.818109:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71315456 i_blocks 139312 00000020:00200000:0.0:1789760778.818114:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.818123:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.818160:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71315456, count: 4096 00000080:00200000:0.0:1789760778.818168:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71315456, bytes: 4096 00000080:00200000:0.0:1789760778.818180:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.818189:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.818191:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71315456, 71319552) 0 0 00000080:00200000:0.0:1789760778.818196:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.818212:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17411, 17411] 00020000:00200000:0.0:1789760778.818221:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71315456 chunk: [71315456, 71319552) 71319552, 4096 00020000:00200000:0.0:1789760778.818226:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.818314:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17411, 17411] 00000008:00010000:0.0:1789760778.818334:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.818339:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.818350:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71315456, 71319552) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.818364:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71315456=0x4403000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.818603:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.818606:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71319552 00000020:00010000:0.0:1789760778.818615:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.818619:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.818622:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.818626:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.818629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.818638:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.818720:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.818738:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.822814:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71315456<>71319552 00000100:00100000:1.0:1789760778.824186:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565458688:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.832744:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984719 gen 3 00000008:00010000:0.0:1789760778.832799:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.832809:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.832821:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.832967:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.832998:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565458688:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.833092:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17411, 17411] 00000080:00200000:1.0:1789760778.833109:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.833115:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.833130:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71319552 i_blocks 139320 00000020:00200000:1.0:1789760778.833135:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.833146:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.833182:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71319552, count: 4096 00000080:00200000:1.0:1789760778.833190:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71319552, bytes: 4096 00000080:00200000:1.0:1789760778.833202:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.833211:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.833214:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71319552, 71323648) 0 0 00000080:00200000:1.0:1789760778.833219:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.833312:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17412, 17412] 00020000:00200000:1.0:1789760778.833323:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71319552 chunk: [71319552, 71323648) 71323648, 4096 00020000:00200000:1.0:1789760778.833329:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.833349:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17412, 17412] 00000008:00010000:1.0:1789760778.833368:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.833374:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.833595:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71319552, 71323648) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.833612:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71319552=0x4404000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.833758:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.833762:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71323648 00000020:00010000:1.0:1789760778.833771:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.833775:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.833779:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.833783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.833786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.833795:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.833802:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.833822:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760778.834046:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 71319552<>71323648 00000100:00100000:0.0:1789760778.834119:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458816:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.845248:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984720 gen 3 00000008:00010000:0.0:1789760778.845304:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760778.845313:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.845323:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.845398:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760778.845428:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565458816:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.845943:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17412, 17412] 00000080:00200000:1.0:1789760778.845961:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.845968:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.845984:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71323648 i_blocks 139328 00000020:00200000:1.0:1789760778.845987:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.845997:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.846035:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71323648, count: 4096 00000080:00200000:1.0:1789760778.846043:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71323648, bytes: 4096 00000080:00200000:1.0:1789760778.846053:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.846062:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.846065:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71323648, 71327744) 0 0 00000080:00200000:1.0:1789760778.846070:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.846087:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17413, 17413] 00020000:00200000:1.0:1789760778.846096:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71323648 chunk: [71323648, 71327744) 71327744, 4096 00020000:00200000:1.0:1789760778.846101:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.846123:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17413, 17413] 00000008:00010000:1.0:1789760778.846142:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.846148:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.846161:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71323648, 71327744) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.846175:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71323648=0x4405000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.846232:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.846235:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71327744 00000020:00010000:1.0:1789760778.846244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.846248:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.846251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.846255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.846257:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.846265:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.846271:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.846291:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760778.846481:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 71323648<>71327744 00000100:00100000:0.0:1789760778.846671:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565458944:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.855214:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984721 gen 3 00000008:00010000:0.0:1789760778.855267:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760778.855276:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.855288:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.855308:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760778.855340:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565458944:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.855613:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17413, 17413] 00000080:00200000:1.0:1789760778.855631:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.855637:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.855693:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71327744 i_blocks 139336 00000020:00200000:1.0:1789760778.855698:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.855708:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.855745:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71327744, count: 4096 00000080:00200000:1.0:1789760778.855753:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71327744, bytes: 4096 00000080:00200000:1.0:1789760778.855764:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.855773:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.855775:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71327744, 71331840) 0 0 00000080:00200000:1.0:1789760778.855780:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.855796:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17414, 17414] 00020000:00200000:1.0:1789760778.855804:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71327744 chunk: [71327744, 71331840) 71331840, 4096 00020000:00200000:1.0:1789760778.855809:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.855829:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17414, 17414] 00000008:00010000:1.0:1789760778.855849:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.855856:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.855868:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71327744, 71331840) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.855882:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71327744=0x4406000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.855943:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.855946:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71331840 00000020:00010000:1.0:1789760778.855957:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.855960:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.855963:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.855967:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.855970:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.855977:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.855984:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.856001:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760778.856138:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 71327744<>71331840 00000100:00100000:0.0:1789760778.856927:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565459072:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.865818:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984722 gen 3 00000008:00010000:0.0:1789760778.865869:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760778.865878:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.865888:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.865910:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760778.865938:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565459072:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.866084:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17414, 17414] 00000080:00200000:1.0:1789760778.866101:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.866107:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.866121:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71331840 i_blocks 139344 00000020:00200000:1.0:1789760778.866125:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.866135:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.866168:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71331840, count: 4096 00000080:00200000:1.0:1789760778.866176:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71331840, bytes: 4096 00000080:00200000:1.0:1789760778.866197:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.866206:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.866209:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71331840, 71335936) 0 0 00000080:00200000:1.0:1789760778.866215:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.866232:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17415, 17415] 00020000:00200000:1.0:1789760778.866249:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71331840 chunk: [71331840, 71335936) 71335936, 4096 00020000:00200000:1.0:1789760778.866255:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.866275:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17415, 17415] 00000008:00010000:1.0:1789760778.866294:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.866300:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.866312:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71331840, 71335936) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.866326:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71331840=0x4407000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.866598:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.866601:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71335936 00000020:00010000:1.0:1789760778.866611:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.866614:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.866618:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.866621:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.866625:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.866632:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.866639:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.866686:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760778.866806:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 71331840<>71335936 00000100:00100000:0.0:1789760778.866884:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565459200:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.876117:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984723 gen 3 00000008:00010000:0.0:1789760778.876172:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760778.876183:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.876193:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.876215:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760778.876247:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565459200:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.876357:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17415, 17415] 00000080:00200000:1.0:1789760778.876375:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.876381:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.876398:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71335936 i_blocks 139352 00000020:00200000:1.0:1789760778.876402:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.876412:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.876458:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71335936, count: 4096 00000080:00200000:1.0:1789760778.876466:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71335936, bytes: 4096 00000080:00200000:1.0:1789760778.876479:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.876613:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.876616:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71335936, 71340032) 0 0 00000080:00200000:1.0:1789760778.876622:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.876640:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17416, 17416] 00020000:00200000:1.0:1789760778.876688:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71335936 chunk: [71335936, 71340032) 71340032, 4096 00020000:00200000:1.0:1789760778.876693:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760778.876718:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17416, 17416] 00000008:00010000:1.0:1789760778.876741:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760778.876748:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760778.876762:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71335936, 71340032) append 0 append lockpos 0 00000080:00200000:1.0:1789760778.876779:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71335936=0x4408000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760778.876842:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760778.876845:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71340032 00000020:00010000:1.0:1789760778.876930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760778.876934:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.876937:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760778.876940:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760778.876943:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760778.876952:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760778.876959:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760778.876978:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760778.877120:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 71335936<>71340032 00000100:00100000:0.0:1789760778.877235:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565459328:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.889233:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984724 gen 3 00000008:00010000:0.0:1789760778.889284:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760778.889292:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.889302:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.889325:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760778.889356:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565459328:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760778.889496:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17416, 17416] 00000080:00200000:1.0:1789760778.889521:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760778.889527:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760778.889550:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71340032 i_blocks 139360 00000020:00200000:1.0:1789760778.889554:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760778.889564:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760778.889608:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71340032, count: 4096 00000080:00200000:1.0:1789760778.889616:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71340032, bytes: 4096 00000080:00200000:1.0:1789760778.889626:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760778.889633:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760778.889636:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71340032, 71344128) 0 0 00000080:00200000:1.0:1789760778.889641:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760778.889699:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17417, 17417] 00020000:00200000:1.0:1789760778.889814:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71340032 chunk: [71340032, 71344128) 71344128, 4096 00020000:00200000:1.0:1789760778.889820:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:3.0:1789760778.891239:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565459456:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.905928:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17417, 17417] 00000080:00200000:0.0:1789760778.905946:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.905952:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.905967:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71344128 i_blocks 139368 00000020:00200000:0.0:1789760778.905971:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.905980:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.906017:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71344128, count: 4096 00000080:00200000:0.0:1789760778.906024:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71344128, bytes: 4096 00000080:00200000:0.0:1789760778.906035:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.906044:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.906047:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71344128, 71348224) 0 0 00000080:00200000:0.0:1789760778.906051:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.906080:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17418, 17418] 00020000:00200000:0.0:1789760778.906089:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71344128 chunk: [71344128, 71348224) 71348224, 4096 00020000:00200000:0.0:1789760778.906094:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.906114:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17418, 17418] 00000008:00010000:0.0:1789760778.906132:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.906138:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.906151:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71344128, 71348224) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.906164:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71344128=0x440a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.906223:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.906227:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71348224 00000020:00010000:0.0:1789760778.906234:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.906238:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.906241:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.906244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.906247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.906255:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.906260:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.906277:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.906406:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71344128<>71348224 00000100:00100000:1.0:1789760778.906484:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565459584:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.920886:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984726 gen 3 00000008:00010000:0.0:1789760778.920940:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.920948:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.920961:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.921237:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.921267:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565459584:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.922277:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565459712:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.931017:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984727 gen 3 00000008:00010000:3.0:1789760778.931067:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760778.931076:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.931086:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.931108:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760778.931137:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565459712:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.931139:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17419, 17419] 00000080:00200000:0.0:1789760778.931159:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.931165:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.931181:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71352320 i_blocks 139384 00000020:00200000:0.0:1789760778.931185:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.931196:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.931229:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71352320, count: 4096 00000080:00200000:0.0:1789760778.931236:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71352320, bytes: 4096 00000080:00200000:0.0:1789760778.931247:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.931255:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.931258:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71352320, 71356416) 0 0 00000080:00200000:0.0:1789760778.931262:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.931281:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17420, 17420] 00020000:00200000:0.0:1789760778.931290:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71352320 chunk: [71352320, 71356416) 71356416, 4096 00020000:00200000:0.0:1789760778.931295:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.931316:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17420, 17420] 00000008:00010000:0.0:1789760778.931408:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.931415:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.931428:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71352320, 71356416) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.931443:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71352320=0x440c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.931517:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.931520:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71356416 00000020:00010000:0.0:1789760778.931529:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.931533:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.931536:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.931539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.931542:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.931552:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.931559:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.931575:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.931735:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71352320<>71356416 00000100:00100000:0.0:1789760778.931785:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760778.931801:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565459840:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760778.945278:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984728 gen 3 00000008:00010000:0.0:1789760778.945335:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760778.945343:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760778.945355:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760778.945389:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760778.945420:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565459840:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.946398:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565459968:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.959117:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984729 gen 3 00000008:00010000:3.0:1789760778.959172:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760778.959181:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.959193:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.959219:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760778.959260:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565459968:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.960568:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565460096:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.970229:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984730 gen 3 00000008:00010000:3.0:1789760778.970281:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760778.970290:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.970300:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.970323:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760778.970354:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565460096:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.971605:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760778.971944:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565460224:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.979391:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984731 gen 3 00000008:00010000:3.0:1789760778.979554:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760778.979565:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760778.979576:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760778.979600:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760778.979966:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565460224:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760778.980663:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565460352:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760778.992284:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17424, 17424] 00000080:00200000:0.0:1789760778.992301:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760778.992307:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760778.992324:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71372800 i_blocks 139424 00000020:00200000:0.0:1789760778.992328:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760778.992340:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760778.992377:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71372800, count: 4096 00000080:00200000:0.0:1789760778.992384:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71372800, bytes: 4096 00000080:00200000:0.0:1789760778.992399:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760778.992409:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760778.992411:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71372800, 71376896) 0 0 00000080:00200000:0.0:1789760778.992416:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760778.992436:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17425, 17425] 00020000:00200000:0.0:1789760778.992555:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71372800 chunk: [71372800, 71376896) 71376896, 4096 00020000:00200000:0.0:1789760778.992561:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760778.992582:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17425, 17425] 00000008:00010000:0.0:1789760778.992600:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760778.992606:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760778.992619:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71372800, 71376896) append 0 append lockpos 0 00000080:00200000:0.0:1789760778.992817:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71372800=0x4411000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760778.992960:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760778.992964:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71376896 00000020:00010000:0.0:1789760778.992973:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760778.992977:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.992981:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760778.992985:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760778.992988:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760778.992997:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760778.993004:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760778.993031:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760778.993323:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71372800<>71376896 00000100:00100000:1.0:1789760778.993406:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565460480:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.002531:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984733 gen 3 00000008:00010000:0.0:1789760779.002587:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760779.002596:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.002609:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.002637:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760779.004316:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565460480:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.016304:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17426, 17426] 00000080:00200000:0.0:1789760779.016322:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.016328:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.016347:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71380992 i_blocks 139440 00000020:00200000:0.0:1789760779.016351:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.016362:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.016398:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71380992, count: 4096 00000080:00200000:0.0:1789760779.016406:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71380992, bytes: 4096 00000080:00200000:0.0:1789760779.016419:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.016427:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.016430:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71380992, 71385088) 0 0 00000080:00200000:0.0:1789760779.016435:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.016463:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17427, 17427] 00020000:00200000:0.0:1789760779.016472:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71380992 chunk: [71380992, 71385088) 71385088, 4096 00020000:00200000:0.0:1789760779.016478:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.016516:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17427, 17427] 00000008:00010000:0.0:1789760779.016537:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.016544:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.016557:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71380992, 71385088) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.016571:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71380992=0x4413000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.016833:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.016836:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71385088 00000020:00010000:0.0:1789760779.016846:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.016850:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.016853:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.016857:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.016871:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.016881:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.016888:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.016908:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760779.017043:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71380992<>71385088 00000100:00100000:0.0:1789760779.017087:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760779.017106:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565460736:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.027041:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984735 gen 3 00000008:00010000:0.0:1789760779.027091:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760779.027100:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.027110:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.027139:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760779.027169:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565460736:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.029513:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760779.029539:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565460864:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.037274:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984736 gen 3 00000008:00010000:3.0:1789760779.037313:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760779.037319:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.037327:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.037347:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760779.037372:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565460864:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.038782:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565460992:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.050969:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984737 gen 3 00000008:00010000:3.0:1789760779.051007:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760779.051014:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.051023:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.051043:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760779.051066:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565460992:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.062273:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17430, 17430] 00000080:00200000:0.0:1789760779.062297:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.062304:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.062320:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71397376 i_blocks 139472 00000020:00200000:0.0:1789760779.062325:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.062339:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.062377:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71397376, count: 4096 00000080:00200000:0.0:1789760779.062386:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71397376, bytes: 4096 00000080:00200000:0.0:1789760779.062400:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.062409:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.062411:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71397376, 71401472) 0 0 00000080:00200000:0.0:1789760779.062416:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.062433:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17431, 17431] 00020000:00200000:0.0:1789760779.062442:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71397376 chunk: [71397376, 71401472) 71401472, 4096 00020000:00200000:0.0:1789760779.062447:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.062468:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17431, 17431] 00000008:00010000:0.0:1789760779.062488:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.062494:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.062516:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71397376, 71401472) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.062533:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71397376=0x4417000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.062603:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.062606:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71401472 00000020:00010000:0.0:1789760779.062614:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.062618:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.062622:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.062625:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.062639:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.062702:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.062709:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.062731:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760779.062874:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71397376<>71401472 00000100:00100000:1.0:1789760779.063166:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760779.063195:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565461248:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.074360:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984739 gen 3 00000008:00010000:0.0:1789760779.074403:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760779.074409:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.074418:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.074441:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760779.074532:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565461248:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.075046:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565461376:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.084916:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984740 gen 3 00000008:00010000:3.0:1789760779.084951:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760779.084957:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.084965:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.084985:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760779.085008:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565461376:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.085738:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565461504:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.097968:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984741 gen 3 00000008:00010000:3.0:1789760779.098003:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760779.098009:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.098018:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.098039:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760779.098064:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565461504:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760779.111767:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17434, 17434] 00000080:00200000:3.0:1789760779.111779:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760779.111783:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760779.111795:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71413760 i_blocks 139504 00000020:00200000:3.0:1789760779.111803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760779.111812:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760779.111840:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71413760, count: 4096 00000080:00200000:3.0:1789760779.111845:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71413760, bytes: 4096 00000080:00200000:3.0:1789760779.111969:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760779.111978:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760779.111980:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71413760, 71417856) 0 0 00000080:00200000:3.0:1789760779.111984:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760779.111997:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17435, 17435] 00020000:00200000:3.0:1789760779.112004:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71413760 chunk: [71413760, 71417856) 71417856, 4096 00020000:00200000:3.0:1789760779.112007:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760779.112023:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17435, 17435] 00000008:00010000:3.0:1789760779.112036:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760779.112041:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760779.112049:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71413760, 71417856) append 0 append lockpos 0 00000080:00200000:3.0:1789760779.112060:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71413760=0x441b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760779.112101:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760779.112104:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71417856 00000020:00010000:3.0:1789760779.112110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760779.112113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760779.112116:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760779.112123:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760779.112126:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760779.112133:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760779.112138:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760779.112153:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760779.112243:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 71413760<>71417856 00000100:00100000:3.0:1789760779.112285:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760779.112299:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565461760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.125992:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984743 gen 3 00000008:00010000:3.0:1789760779.126033:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:3.0:1789760779.126039:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.126048:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.126069:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:3.0:1789760779.126093:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565461760:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.126098:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17435, 17435] 00000080:00200000:0.0:1789760779.126110:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.126115:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.126128:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71417856 i_blocks 139512 00000020:00200000:0.0:1789760779.126131:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.126139:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.126179:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71417856, count: 4096 00000080:00200000:0.0:1789760779.126186:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71417856, bytes: 4096 00000080:00200000:0.0:1789760779.126193:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.126201:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.126203:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71417856, 71421952) 0 0 00000080:00200000:0.0:1789760779.126206:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.126220:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17436, 17436] 00020000:00200000:0.0:1789760779.126228:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71417856 chunk: [71417856, 71421952) 71421952, 4096 00020000:00200000:0.0:1789760779.126232:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.126246:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17436, 17436] 00000008:00010000:0.0:1789760779.126259:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.126264:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.126273:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71417856, 71421952) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.126286:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71417856=0x441c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.126330:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.126333:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71421952 00000020:00010000:0.0:1789760779.126339:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.126342:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.126345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.126349:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.126351:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.126358:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.126364:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.126377:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760779.126480:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 71417856<>71421952 00000100:00100000:1.0:1789760779.126601:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565461888:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.154693:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984744 gen 3 00000008:00010000:1.0:1789760779.154739:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760779.154745:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.154757:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.154786:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000080:00200000:0.0:1789760779.154814:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17436, 17436] 00000100:00100000:1.0:1789760779.154815:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565461888:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.154831:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.154837:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.154916:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71421952 i_blocks 139520 00000020:00200000:0.0:1789760779.154921:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.154930:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.154958:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71421952, count: 4096 00000080:00200000:0.0:1789760779.154965:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71421952, bytes: 4096 00000080:00200000:0.0:1789760779.154972:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.154980:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.154981:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71421952, 71426048) 0 0 00000080:00200000:0.0:1789760779.154985:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.155000:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17437, 17437] 00020000:00200000:0.0:1789760779.155007:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71421952 chunk: [71421952, 71426048) 71426048, 4096 00020000:00200000:0.0:1789760779.155010:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.155025:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17437, 17437] 00000008:00010000:0.0:1789760779.155041:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.155046:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.155055:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71421952, 71426048) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.155068:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71421952=0x441d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.155111:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.155113:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71426048 00000020:00010000:0.0:1789760779.155120:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.155124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.155126:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.155130:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.155133:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.155140:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.155156:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.155171:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760779.155267:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 71421952<>71426048 00000100:00100000:1.0:1789760779.155660:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760779.155677:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565462016:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.168252:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984745 gen 3 00000008:00010000:1.0:1789760779.168297:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760779.168303:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.168312:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.168330:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760779.168353:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565462016:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.168455:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17437, 17437] 00000080:00200000:0.0:1789760779.168468:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.168473:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.168498:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71426048 i_blocks 139528 00000020:00200000:0.0:1789760779.168501:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.168518:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.168544:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71426048, count: 4096 00000080:00200000:0.0:1789760779.168551:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71426048, bytes: 4096 00000080:00200000:0.0:1789760779.168558:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.168565:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.168568:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71426048, 71430144) 0 0 00000080:00200000:0.0:1789760779.168572:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.168585:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17438, 17438] 00020000:00200000:0.0:1789760779.168593:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71426048 chunk: [71426048, 71430144) 71430144, 4096 00020000:00200000:0.0:1789760779.168596:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.168612:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17438, 17438] 00000008:00010000:0.0:1789760779.168628:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.168633:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.168669:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71426048, 71430144) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.168681:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71426048=0x441e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.168725:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.168728:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71430144 00000020:00010000:0.0:1789760779.168735:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.168738:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.168741:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.168745:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.168747:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.168754:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.168760:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.168775:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.168876:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 71426048<>71430144 00000100:00100000:1.0:1789760779.168937:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565462144:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.183292:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984746 gen 3 00000008:00010000:1.0:1789760779.183329:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760779.183335:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.183343:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.183360:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760779.183385:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565462144:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.183754:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17438, 17438] 00000080:00200000:0.0:1789760779.183766:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.183771:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.183781:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71430144 i_blocks 139536 00000020:00200000:0.0:1789760779.183785:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.183793:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.183819:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71430144, count: 4096 00000080:00200000:0.0:1789760779.183825:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71430144, bytes: 4096 00000080:00200000:0.0:1789760779.183834:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.183841:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.183844:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71430144, 71434240) 0 0 00000080:00200000:0.0:1789760779.183931:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.183944:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17439, 17439] 00020000:00200000:0.0:1789760779.183952:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71430144 chunk: [71430144, 71434240) 71434240, 4096 00020000:00200000:0.0:1789760779.183956:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.183972:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17439, 17439] 00000008:00010000:0.0:1789760779.183985:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.183990:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.183998:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71430144, 71434240) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.184008:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71430144=0x441f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.184051:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.184053:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71434240 00000020:00010000:0.0:1789760779.184060:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.184063:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.184066:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.184070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.184073:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.184080:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.184085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.184098:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760779.184194:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 71430144<>71434240 00000100:00100000:1.0:1789760779.184256:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565462272:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.199967:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984747 gen 3 00000008:00010000:1.0:1789760779.200006:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760779.200012:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.200021:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.200080:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:1.0:1789760779.200105:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565462272:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.200314:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17439, 17439] 00000080:00200000:0.0:1789760779.200327:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.200332:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.200345:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71434240 i_blocks 139544 00000020:00200000:0.0:1789760779.200349:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.200358:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.200383:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71434240, count: 4096 00000080:00200000:0.0:1789760779.200390:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71434240, bytes: 4096 00000080:00200000:0.0:1789760779.200397:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.200406:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.200408:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71434240, 71438336) 0 0 00000080:00200000:0.0:1789760779.200412:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.200426:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17440, 17440] 00020000:00200000:0.0:1789760779.200432:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71434240 chunk: [71434240, 71438336) 71438336, 4096 00020000:00200000:0.0:1789760779.200436:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.201030:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17440, 17440] 00000008:00010000:0.0:1789760779.201046:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.201051:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.201060:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71434240, 71438336) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.201072:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71434240=0x4420000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.201118:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.201121:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71438336 00000020:00010000:0.0:1789760779.201127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.201130:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.201132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.201136:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.201138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.201221:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.201229:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.201316:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760779.201410:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41f80 - object 0x240000400:5 offset 71434240<>71438336 00000100:00100000:1.0:1789760779.201494:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565462400:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.215537:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984748 gen 3 00000008:00010000:1.0:1789760779.215571:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760779.215577:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.215585:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.215603:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760779.215631:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565462400:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.215730:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17440, 17440] 00000080:00200000:0.0:1789760779.215742:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.215747:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.215758:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71438336 i_blocks 139552 00000020:00200000:0.0:1789760779.215761:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.215769:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.215790:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71438336, count: 4096 00000080:00200000:0.0:1789760779.215849:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71438336, bytes: 4096 00000080:00200000:0.0:1789760779.215857:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.215864:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.215866:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71438336, 71442432) 0 0 00000080:00200000:0.0:1789760779.215869:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.215982:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17441, 17441] 00020000:00200000:0.0:1789760779.215989:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71438336 chunk: [71438336, 71442432) 71442432, 4096 00020000:00200000:0.0:1789760779.215992:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.216007:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17441, 17441] 00000008:00010000:0.0:1789760779.216017:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.216021:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.216029:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71438336, 71442432) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.216038:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71438336=0x4421000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.216074:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.216076:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71442432 00000020:00010000:0.0:1789760779.216082:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.216085:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.216087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.216091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.216093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.216099:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.216103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.216115:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760779.216270:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43100 - object 0x240000400:5 offset 71438336<>71442432 00000100:00100000:1.0:1789760779.216361:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565462528:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.231294:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984749 gen 3 00000008:00010000:1.0:1789760779.231335:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760779.231342:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.231351:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.231369:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:1.0:1789760779.231394:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565462528:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.231635:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17441, 17441] 00000080:00200000:0.0:1789760779.231677:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.231681:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.231690:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71442432 i_blocks 139560 00000020:00200000:0.0:1789760779.231692:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.231700:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.231722:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71442432, count: 4096 00000080:00200000:0.0:1789760779.231727:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71442432, bytes: 4096 00000080:00200000:0.0:1789760779.231734:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.231740:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.231741:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71442432, 71446528) 0 0 00000080:00200000:0.0:1789760779.231744:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.231755:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17442, 17442] 00020000:00200000:0.0:1789760779.231761:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71442432 chunk: [71442432, 71446528) 71446528, 4096 00020000:00200000:0.0:1789760779.231763:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.231775:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17442, 17442] 00000008:00010000:0.0:1789760779.231785:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.231789:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.231796:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71442432, 71446528) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.231806:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71442432=0x4422000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.231844:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.231847:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71446528 00000020:00010000:0.0:1789760779.231866:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.231869:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.231873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.231876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.231879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.231885:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.231891:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760779.231947:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:1.0:1789760779.232045:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 71442432<>71446528 00000100:00100000:1.0:1789760779.232093:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565462656:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.243121:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984750 gen 3 00000008:00010000:1.0:1789760779.243155:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760779.243161:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.243181:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.243197:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760779.243220:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565462656:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.243383:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17442, 17442] 00000080:00200000:0.0:1789760779.243396:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.243401:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.243413:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71446528 i_blocks 139568 00000020:00200000:0.0:1789760779.243416:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.243426:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.243449:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71446528, count: 4096 00000080:00200000:0.0:1789760779.243455:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71446528, bytes: 4096 00000080:00200000:0.0:1789760779.243463:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.243470:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.243472:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71446528, 71450624) 0 0 00000080:00200000:0.0:1789760779.243476:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.243837:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17443, 17443] 00020000:00200000:0.0:1789760779.243845:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71446528 chunk: [71446528, 71450624) 71450624, 4096 00020000:00200000:0.0:1789760779.243849:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.243864:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17443, 17443] 00000008:00010000:0.0:1789760779.243879:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.244205:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.244218:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71446528, 71450624) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.244228:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71446528=0x4423000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.244288:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.244292:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71450624 00000020:00010000:0.0:1789760779.244299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.244302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.244305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.244309:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.244312:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.244320:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.244326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.244344:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:0.0:1789760779.244442:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40700 - object 0x240000400:5 offset 71446528<>71450624 00000100:00100000:1.0:1789760779.244693:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565462784:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.257121:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984751 gen 3 00000008:00010000:1.0:1789760779.257174:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760779.257183:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.257195:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.257215:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760779.257256:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565462784:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.257412:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17443, 17443] 00000080:00200000:0.0:1789760779.257428:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.257435:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.257565:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71450624 i_blocks 139576 00000020:00200000:0.0:1789760779.257570:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.257582:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.257617:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71450624, count: 4096 00000080:00200000:0.0:1789760779.257624:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71450624, bytes: 4096 00000080:00200000:0.0:1789760779.257635:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.257686:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.257689:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71450624, 71454720) 0 0 00000080:00200000:0.0:1789760779.257693:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.257712:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17444, 17444] 00020000:00200000:0.0:1789760779.257720:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71450624 chunk: [71450624, 71454720) 71454720, 4096 00020000:00200000:0.0:1789760779.257725:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.257747:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17444, 17444] 00000008:00010000:0.0:1789760779.257766:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.257772:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.257783:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71450624, 71454720) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.257797:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71450624=0x4424000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.257855:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.257858:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71454720 00000020:00010000:0.0:1789760779.257866:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.257870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.257873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.257876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.258197:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.258211:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.258219:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.258305:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760779.258425:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 71450624<>71454720 00000100:00100000:1.0:1789760779.258614:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565462912:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.272783:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984752 gen 3 00000008:00010000:1.0:1789760779.272835:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760779.272844:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.272854:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.272886:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:1.0:1789760779.272918:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565462912:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760779.273069:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17444, 17444] 00000080:00200000:3.0:1789760779.273086:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760779.273093:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760779.273108:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71454720 i_blocks 139584 00000020:00200000:3.0:1789760779.273111:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760779.273122:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760779.273159:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71454720, count: 4096 00000080:00200000:3.0:1789760779.273167:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71454720, bytes: 4096 00000080:00200000:3.0:1789760779.273178:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760779.273187:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760779.273190:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71454720, 71458816) 0 0 00000080:00200000:3.0:1789760779.273194:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760779.273212:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17445, 17445] 00020000:00200000:3.0:1789760779.273220:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71454720 chunk: [71454720, 71458816) 71458816, 4096 00020000:00200000:3.0:1789760779.273225:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760779.273256:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17445, 17445] 00000008:00010000:3.0:1789760779.273274:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760779.273280:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760779.273292:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71454720, 71458816) append 0 append lockpos 0 00000080:00200000:3.0:1789760779.273306:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71454720=0x4425000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760779.273365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760779.273368:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71458816 00000020:00010000:3.0:1789760779.273375:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760779.273378:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760779.273381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760779.273385:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760779.273388:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760779.273395:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760779.273402:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760779.273417:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760779.273641:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 71454720<>71458816 00000080:00200000:3.0:1789760779.283599:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17445, 17445] 00000080:00200000:3.0:1789760779.283618:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760779.283625:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760779.283674:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71458816 i_blocks 139592 00000020:00200000:3.0:1789760779.283678:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760779.283690:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760779.283728:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71458816, count: 4096 00000080:00200000:3.0:1789760779.283737:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71458816, bytes: 4096 00000080:00200000:3.0:1789760779.283750:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760779.283761:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760779.283763:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71458816, 71462912) 0 0 00000080:00200000:3.0:1789760779.283768:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760779.283789:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17446, 17446] 00020000:00200000:3.0:1789760779.283799:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71458816 chunk: [71458816, 71462912) 71462912, 4096 00020000:00200000:3.0:1789760779.283803:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760779.283827:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17446, 17446] 00000008:00010000:3.0:1789760779.283862:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760779.283868:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760779.283881:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71458816, 71462912) append 0 append lockpos 0 00000080:00200000:3.0:1789760779.283899:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71458816=0x4426000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760779.283959:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760779.283961:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71462912 00000020:00010000:3.0:1789760779.283970:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760779.283973:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760779.283976:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760779.283979:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760779.283982:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760779.283990:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760779.283998:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760779.284016:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760779.284148:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 71458816<>71462912 00000100:00100000:3.0:1789760779.284202:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565463168:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.296198:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984754 gen 3 00000008:00010000:3.0:1789760779.296252:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:3.0:1789760779.296262:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.296274:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.296307:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:3.0:1789760779.296344:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565463168:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760779.297594:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17446, 17446] 00000080:00200000:1.0:1789760779.297618:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760779.297624:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760779.297641:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71462912 i_blocks 139600 00000020:00200000:1.0:1789760779.297664:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760779.297826:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760779.297867:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71462912, count: 4096 00000080:00200000:1.0:1789760779.297875:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71462912, bytes: 4096 00000080:00200000:1.0:1789760779.297890:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760779.297899:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760779.297902:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71462912, 71467008) 0 0 00000080:00200000:1.0:1789760779.297907:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760779.297925:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17447, 17447] 00020000:00200000:1.0:1789760779.298002:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71462912 chunk: [71462912, 71467008) 71467008, 4096 00020000:00200000:1.0:1789760779.298008:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760779.298039:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17447, 17447] 00000008:00010000:1.0:1789760779.298059:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760779.298066:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760779.298079:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71462912, 71467008) append 0 append lockpos 0 00000080:00200000:1.0:1789760779.298266:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71462912=0x4427000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760779.298338:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760779.298341:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71467008 00000020:00010000:1.0:1789760779.298350:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760779.298354:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.298357:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.298360:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760779.298363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760779.298383:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760779.298390:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760779.298415:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760779.299252:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 71462912<>71467008 00000100:00100000:1.0:1789760779.299316:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760779.299343:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565463296:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.314625:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984755 gen 3 00000008:00010000:1.0:1789760779.314693:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:1.0:1789760779.314703:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.314715:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.314750:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:1.0:1789760779.314779:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565463296:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760779.314919:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17447, 17447] 00000080:00200000:3.0:1789760779.314935:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760779.314940:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760779.314955:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71467008 i_blocks 139608 00000020:00200000:3.0:1789760779.314958:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760779.314968:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760779.315002:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71467008, count: 4096 00000080:00200000:3.0:1789760779.315009:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71467008, bytes: 4096 00000080:00200000:3.0:1789760779.315019:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760779.315028:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760779.315030:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71467008, 71471104) 0 0 00000080:00200000:3.0:1789760779.315034:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760779.315053:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17448, 17448] 00020000:00200000:3.0:1789760779.315062:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71467008 chunk: [71467008, 71471104) 71471104, 4096 00020000:00200000:3.0:1789760779.315067:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760779.315087:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17448, 17448] 00000008:00010000:3.0:1789760779.315106:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760779.315112:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760779.315124:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71467008, 71471104) append 0 append lockpos 0 00000080:00200000:3.0:1789760779.315139:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71467008=0x4428000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760779.315196:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760779.315199:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71471104 00000020:00010000:3.0:1789760779.315207:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760779.315210:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760779.315214:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760779.315217:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760779.315219:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760779.315228:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760779.315235:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760779.315252:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760779.315374:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 71467008<>71471104 00000100:00100000:3.0:1789760779.315422:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760779.315437:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565463424:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760779.336351:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17448, 17448] 00000080:00200000:1.0:1789760779.336370:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760779.336377:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760779.336395:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71471104 i_blocks 139616 00000020:00200000:1.0:1789760779.336399:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760779.336411:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760779.336460:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71471104, count: 4096 00000080:00200000:1.0:1789760779.336468:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71471104, bytes: 4096 00000080:00200000:1.0:1789760779.336480:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760779.336489:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760779.336492:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71471104, 71475200) 0 0 00000080:00200000:1.0:1789760779.336496:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760779.336522:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17449, 17449] 00020000:00200000:1.0:1789760779.336532:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71471104 chunk: [71471104, 71475200) 71475200, 4096 00020000:00200000:1.0:1789760779.336537:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760779.336558:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17449, 17449] 00000008:00010000:1.0:1789760779.336577:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760779.336584:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760779.336596:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71471104, 71475200) append 0 append lockpos 0 00000080:00200000:1.0:1789760779.336610:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71471104=0x4429000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760779.337756:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760779.337765:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71475200 00000020:00010000:1.0:1789760779.337778:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760779.337783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.337786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.337790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760779.337793:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760779.337802:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760779.337810:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760779.337825:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760779.339311:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 71471104<>71475200 00000100:00100000:0.0:1789760779.339382:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760779.339665:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565463552:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.351097:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984757 gen 3 00000008:00010000:1.0:1789760779.351163:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:1.0:1789760779.351175:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.351190:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.351216:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:1.0:1789760779.351249:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565463552:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.351365:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17449, 17449] 00000080:00200000:0.0:1789760779.351390:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.351399:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.351417:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71475200 i_blocks 139624 00000020:00200000:0.0:1789760779.351420:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.351433:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.351478:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71475200, count: 4096 00000080:00200000:0.0:1789760779.351486:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71475200, bytes: 4096 00000080:00200000:0.0:1789760779.351497:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.351517:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.351520:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71475200, 71479296) 0 0 00000080:00200000:0.0:1789760779.351525:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.351542:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17450, 17450] 00020000:00200000:0.0:1789760779.351551:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71475200 chunk: [71475200, 71479296) 71479296, 4096 00020000:00200000:0.0:1789760779.351556:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.351904:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17450, 17450] 00000008:00010000:0.0:1789760779.351926:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.351932:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.351945:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71475200, 71479296) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.351959:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71475200=0x442a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.352025:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.352027:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71479296 00000020:00010000:0.0:1789760779.352037:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.352041:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.352044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.352049:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.352052:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.352061:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.352068:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.352088:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760779.352296:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42680 - object 0x240000400:5 offset 71475200<>71479296 00000100:00100000:1.0:1789760779.352413:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565463680:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.371205:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984758 gen 3 00000008:00010000:1.0:1789760779.371255:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760779.371271:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.371283:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.371364:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760779.371393:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565463680:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.372887:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17450, 17450] 00000080:00200000:0.0:1789760779.372909:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.372915:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.372932:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71479296 i_blocks 139632 00000020:00200000:0.0:1789760779.372937:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.372953:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.372985:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71479296, count: 4096 00000080:00200000:0.0:1789760779.372992:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71479296, bytes: 4096 00000080:00200000:0.0:1789760779.373151:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.373162:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.373165:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71479296, 71483392) 0 0 00000080:00200000:0.0:1789760779.373170:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.373186:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17451, 17451] 00020000:00200000:0.0:1789760779.373195:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71479296 chunk: [71479296, 71483392) 71483392, 4096 00020000:00200000:0.0:1789760779.373199:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.373226:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17451, 17451] 00000008:00010000:0.0:1789760779.373244:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.373249:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.373343:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71479296, 71483392) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.373358:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71479296=0x442b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.373426:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.373429:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71483392 00000020:00010000:0.0:1789760779.373489:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.373494:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.373497:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.373510:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.373513:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.373523:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.373531:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.373559:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760779.374928:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 71479296<>71483392 00000100:00100000:0.0:1789760779.376391:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760779.376432:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565463808:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.391456:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984759 gen 3 00000008:00010000:1.0:1789760779.391518:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760779.391527:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.391538:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.391573:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:1.0:1789760779.391606:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565463808:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760779.404244:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17452, 17452] 00000080:00200000:1.0:1789760779.404261:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760779.404267:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760779.404282:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71487488 i_blocks 139648 00000020:00200000:1.0:1789760779.404286:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760779.404296:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760779.404330:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71487488, count: 4096 00000080:00200000:1.0:1789760779.404338:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71487488, bytes: 4096 00000080:00200000:1.0:1789760779.404350:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760779.404358:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760779.404361:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71487488, 71491584) 0 0 00000080:00200000:1.0:1789760779.404366:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760779.404382:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17453, 17453] 00020000:00200000:1.0:1789760779.404391:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71487488 chunk: [71487488, 71491584) 71491584, 4096 00020000:00200000:1.0:1789760779.404396:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760779.404415:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17453, 17453] 00000008:00010000:1.0:1789760779.404434:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760779.404513:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760779.404528:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71487488, 71491584) append 0 append lockpos 0 00000080:00200000:1.0:1789760779.404543:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71487488=0x442d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760779.404862:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760779.404865:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71491584 00000020:00010000:1.0:1789760779.404875:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760779.404879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.404882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.404885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760779.404889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760779.404898:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760779.404905:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760779.404927:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760779.405056:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 71487488<>71491584 00000100:00100000:0.0:1789760779.405128:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565464064:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.418114:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984761 gen 3 00000008:00010000:1.0:1789760779.418163:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:1.0:1789760779.418171:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.418183:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.418207:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:1.0:1789760779.418845:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565464064:192.168.203.147@tcp:4:dd.0 00000008:00010000:1.0:1789760779.418899:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189a80 00000008:00100000:1.0:1789760779.419022:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 71491584<>71495680 00000100:00100000:1.0:1789760779.419293:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565464192:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760779.438419:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984762 gen 3 00000008:00010000:1.0:1789760779.438470:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:1.0:1789760779.438479:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760779.438490:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760779.438522:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:1.0:1789760779.438554:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565464192:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.439218:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760779.439237:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565464320:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.448909:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984763 gen 3 00000080:00200000:1.0:1789760779.449481:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17455, 17455] 00000080:00200000:1.0:1789760779.449497:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760779.449514:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760779.449529:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71499776 i_blocks 139672 00000020:00200000:1.0:1789760779.449533:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760779.449542:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760779.449577:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71499776, count: 4096 00000080:00200000:1.0:1789760779.449584:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71499776, bytes: 4096 00000080:00200000:1.0:1789760779.449600:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760779.449609:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760779.449612:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71499776, 71503872) 0 0 00000080:00200000:1.0:1789760779.449616:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760779.449635:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17456, 17456] 00020000:00200000:1.0:1789760779.449662:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71499776 chunk: [71499776, 71503872) 71503872, 4096 00020000:00200000:1.0:1789760779.449668:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760779.449689:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17456, 17456] 00000008:00010000:1.0:1789760779.449707:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760779.449713:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760779.449725:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71499776, 71503872) append 0 append lockpos 0 00000080:00200000:1.0:1789760779.449739:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71499776=0x4430000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760779.449795:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760779.449798:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71503872 00000020:00010000:1.0:1789760779.449806:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760779.449809:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.449812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.449816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760779.449819:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760779.449826:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760779.449833:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760779.449863:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760779.449992:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 71499776<>71503872 00000100:00100000:1.0:1789760779.450290:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760779.450309:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565464448:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.459374:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984764 gen 3 00000008:00010000:0.0:1789760779.459422:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760779.459431:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.459443:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.459509:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760779.459545:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565464448:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760779.459680:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17456, 17456] 00000080:00200000:1.0:1789760779.459697:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760779.459703:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760779.459717:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71503872 i_blocks 139680 00000020:00200000:1.0:1789760779.459721:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760779.459731:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760779.459761:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71503872, count: 4096 00000080:00200000:1.0:1789760779.459768:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71503872, bytes: 4096 00000080:00200000:1.0:1789760779.459777:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760779.459787:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760779.459790:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71503872, 71507968) 0 0 00000080:00200000:1.0:1789760779.459794:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760779.459811:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17457, 17457] 00020000:00200000:1.0:1789760779.459819:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71503872 chunk: [71503872, 71507968) 71507968, 4096 00020000:00200000:1.0:1789760779.459824:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760779.459843:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17457, 17457] 00000008:00010000:1.0:1789760779.459860:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760779.459866:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760779.459938:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71503872, 71507968) append 0 append lockpos 0 00000080:00200000:1.0:1789760779.459953:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71503872=0x4431000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760779.460009:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760779.460012:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71507968 00000020:00010000:1.0:1789760779.460021:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760779.460024:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.460028:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.460032:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760779.460035:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760779.460042:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760779.460049:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760779.460065:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760779.460178:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 71503872<>71507968 00000100:00100000:0.0:1789760779.460242:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565464576:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.468451:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984765 gen 3 00000008:00010000:0.0:1789760779.468511:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760779.468519:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.468530:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.468593:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760779.468633:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565464576:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760779.468869:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17457, 17457] 00000080:00200000:1.0:1789760779.468946:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760779.468954:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760779.468973:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71507968 i_blocks 139688 00000020:00200000:1.0:1789760779.468977:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760779.468988:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760779.469025:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71507968, count: 4096 00000080:00200000:1.0:1789760779.469034:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71507968, bytes: 4096 00000080:00200000:1.0:1789760779.469046:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760779.469056:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760779.469059:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71507968, 71512064) 0 0 00000080:00200000:1.0:1789760779.469064:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760779.469082:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17458, 17458] 00020000:00200000:1.0:1789760779.469092:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71507968 chunk: [71507968, 71512064) 71512064, 4096 00020000:00200000:1.0:1789760779.469097:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760779.469117:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17458, 17458] 00000008:00010000:1.0:1789760779.469133:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760779.469140:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760779.469151:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71507968, 71512064) append 0 append lockpos 0 00000080:00200000:1.0:1789760779.469164:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71507968=0x4432000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760779.469285:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760779.469288:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71512064 00000020:00010000:1.0:1789760779.469298:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760779.469302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.469305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.469309:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760779.469312:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760779.469319:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760779.469326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760779.469345:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760779.469461:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 71507968<>71512064 00000100:00100000:0.0:1789760779.469541:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565464704:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.480579:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984766 gen 3 00000008:00010000:0.0:1789760779.480631:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760779.480640:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.480680:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.480755:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760779.480785:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565464704:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760779.480973:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17458, 17458] 00000080:00200000:1.0:1789760779.480990:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760779.480996:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760779.481078:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71512064 i_blocks 139696 00000020:00200000:1.0:1789760779.481083:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760779.481092:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760779.481127:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71512064, count: 4096 00000080:00200000:1.0:1789760779.481135:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71512064, bytes: 4096 00000080:00200000:1.0:1789760779.481146:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760779.481154:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760779.481157:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71512064, 71516160) 0 0 00000080:00200000:1.0:1789760779.481162:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760779.481179:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17459, 17459] 00020000:00200000:1.0:1789760779.481188:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71512064 chunk: [71512064, 71516160) 71516160, 4096 00020000:00200000:1.0:1789760779.481193:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760779.481213:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17459, 17459] 00000008:00010000:1.0:1789760779.481230:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760779.481236:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760779.481248:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71512064, 71516160) append 0 append lockpos 0 00000080:00200000:1.0:1789760779.481262:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71512064=0x4433000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760779.481323:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760779.481326:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71516160 00000020:00010000:1.0:1789760779.481334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760779.481338:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.481341:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.481345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760779.481348:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760779.481354:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760779.481361:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760779.481379:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760779.481824:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 71512064<>71516160 00000100:00100000:1.0:1789760779.481989:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565464832:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.490824:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984767 gen 3 00000008:00010000:0.0:1789760779.490952:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760779.490963:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.490975:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.491003:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760779.491035:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565464832:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.492953:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760779.492982:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565464960:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.507230:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984768 gen 3 00000008:00010000:3.0:1789760779.507286:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760779.507295:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.507307:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.507336:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760779.507372:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565464960:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.509086:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565465088:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760779.520549:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17461, 17461] 00000080:00200000:3.0:1789760779.520581:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760779.520588:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760779.520916:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71524352 i_blocks 139720 00000020:00200000:3.0:1789760779.520923:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760779.520940:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760779.520973:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71524352, count: 4096 00000080:00200000:3.0:1789760779.520981:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71524352, bytes: 4096 00000080:00200000:3.0:1789760779.520995:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760779.521004:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760779.521006:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71524352, 71528448) 0 0 00000080:00200000:3.0:1789760779.521010:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760779.521029:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17462, 17462] 00020000:00200000:3.0:1789760779.521038:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71524352 chunk: [71524352, 71528448) 71528448, 4096 00020000:00200000:3.0:1789760779.521043:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760779.521065:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17462, 17462] 00000008:00010000:3.0:1789760779.521085:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760779.521092:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760779.521104:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71524352, 71528448) append 0 append lockpos 0 00000080:00200000:3.0:1789760779.521118:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71524352=0x4436000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760779.521185:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760779.521198:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71528448 00000020:00010000:3.0:1789760779.521206:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760779.521210:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760779.521213:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760779.521216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760779.521219:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760779.521229:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760779.521235:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760779.521258:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760779.521387:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 71524352<>71528448 00000080:00200000:3.0:1789760779.533912:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17462, 17462] 00000080:00200000:3.0:1789760779.533929:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760779.533934:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760779.535803:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565465344:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.546698:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17463, 17463] 00000080:00200000:0.0:1789760779.546715:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.546721:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.546735:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71532544 i_blocks 139736 00000020:00200000:0.0:1789760779.546739:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.546749:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.546784:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71532544, count: 4096 00000080:00200000:0.0:1789760779.546793:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71532544, bytes: 4096 00000080:00200000:0.0:1789760779.546805:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.546814:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.546816:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71532544, 71536640) 0 0 00000080:00200000:0.0:1789760779.546821:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.546837:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17464, 17464] 00020000:00200000:0.0:1789760779.547091:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71532544 chunk: [71532544, 71536640) 71536640, 4096 00020000:00200000:0.0:1789760779.547097:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.547121:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17464, 17464] 00000008:00010000:0.0:1789760779.547139:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.547146:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.547158:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71532544, 71536640) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.547173:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71532544=0x4438000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.547246:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.547249:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71536640 00000020:00010000:0.0:1789760779.547259:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.547262:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.547265:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.547268:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.547271:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.547279:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.547616:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.547637:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.547803:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71532544<>71536640 00000100:00100000:0.0:1789760779.547842:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760779.547875:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565465472:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.556487:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984772 gen 3 00000008:00010000:0.0:1789760779.556550:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.556557:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.556568:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.556700:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.556728:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565465472:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.558397:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565465600:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.570420:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17465, 17465] 00000080:00200000:0.0:1789760779.570435:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.570440:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.570455:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71540736 i_blocks 139752 00000020:00200000:0.0:1789760779.570459:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.570469:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.570518:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71540736, count: 4096 00000080:00200000:0.0:1789760779.570526:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71540736, bytes: 4096 00000080:00200000:0.0:1789760779.570537:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.570548:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.570551:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71540736, 71544832) 0 0 00000080:00200000:0.0:1789760779.570557:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.570579:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17466, 17466] 00020000:00200000:0.0:1789760779.570588:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71540736 chunk: [71540736, 71544832) 71544832, 4096 00020000:00200000:0.0:1789760779.570593:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.570610:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17466, 17466] 00000008:00010000:0.0:1789760779.570628:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.570633:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.570670:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71540736, 71544832) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.570684:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71540736=0x443a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.570743:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.570746:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71544832 00000020:00010000:0.0:1789760779.570754:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.570758:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.570766:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.570770:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.570773:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.570781:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.570999:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.571016:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.571136:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71540736<>71544832 00000100:00100000:1.0:1789760779.571199:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565465728:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.584047:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984774 gen 3 00000008:00010000:0.0:1789760779.584097:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.584106:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.584116:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.584197:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.584229:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565465728:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.585250:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565465856:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.597926:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17467, 17467] 00000080:00200000:0.0:1789760779.597943:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.597950:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.598014:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71548928 i_blocks 139768 00000020:00200000:0.0:1789760779.598020:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.598036:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.598077:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71548928, count: 4096 00000080:00200000:0.0:1789760779.598090:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71548928, bytes: 4096 00000080:00200000:0.0:1789760779.598104:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.598113:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.598116:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71548928, 71553024) 0 0 00000080:00200000:0.0:1789760779.598121:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.598141:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17468, 17468] 00020000:00200000:0.0:1789760779.598149:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71548928 chunk: [71548928, 71553024) 71553024, 4096 00020000:00200000:0.0:1789760779.598152:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.598171:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17468, 17468] 00000008:00010000:0.0:1789760779.598187:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.598192:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.598202:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71548928, 71553024) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.598215:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71548928=0x443c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.598279:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.598282:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71553024 00000020:00010000:0.0:1789760779.599392:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.599402:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.599405:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.599409:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.599412:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.599424:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.599434:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.599461:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.599863:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71548928<>71553024 00000100:00100000:1.0:1789760779.599941:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565465984:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.611405:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984776 gen 3 00000008:00010000:0.0:1789760779.611560:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.611569:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.611581:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.611760:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.611780:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565465984:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.616705:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565466112:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.625093:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984777 gen 3 00000008:00010000:3.0:1789760779.625142:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760779.625150:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.625161:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.625186:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760779.625215:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565466112:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.628861:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760779.628890:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565466240:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.638053:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984778 gen 3 00000008:00010000:3.0:1789760779.638107:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760779.638116:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.638127:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.638152:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760779.638180:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565466240:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.639150:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565466368:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.652438:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984779 gen 3 00000008:00010000:3.0:1789760779.652488:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760779.652497:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.652519:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.652601:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760779.652634:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565466368:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.653295:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71565312 i_blocks 139800 00000020:00200000:0.0:1789760779.653302:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.653314:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.653356:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71565312, count: 4096 00000080:00200000:0.0:1789760779.653366:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71565312, bytes: 4096 00000080:00200000:0.0:1789760779.653379:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.653390:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.653393:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71565312, 71569408) 0 0 00000080:00200000:0.0:1789760779.653397:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.653417:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17472, 17472] 00020000:00200000:0.0:1789760779.653427:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71565312 chunk: [71565312, 71569408) 71569408, 4096 00020000:00200000:0.0:1789760779.653432:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.653455:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17472, 17472] 00000008:00010000:0.0:1789760779.653478:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.653485:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.653509:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71565312, 71569408) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.653524:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71565312=0x4440000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.653581:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.653584:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71569408 00000020:00010000:0.0:1789760779.653592:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.653596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.653599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.653602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.653605:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.653613:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.653620:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.653637:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.653793:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71565312<>71569408 00000100:00100000:0.0:1789760779.653853:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760779.653873:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565466496:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.671603:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984780 gen 3 00000008:00010000:0.0:1789760779.671675:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.671684:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.671696:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.671723:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.671753:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565466496:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760779.672340:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17472, 17472] 00000080:00200000:1.0:1789760779.672363:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760779.672370:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760779.672390:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71569408 i_blocks 139808 00000020:00200000:1.0:1789760779.672395:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760779.672409:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760779.672444:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71569408, count: 4096 00000080:00200000:1.0:1789760779.672453:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71569408, bytes: 4096 00000080:00200000:1.0:1789760779.672467:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760779.672476:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760779.672479:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71569408, 71573504) 0 0 00000080:00200000:1.0:1789760779.672484:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760779.672511:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17473, 17473] 00020000:00200000:1.0:1789760779.672521:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71569408 chunk: [71569408, 71573504) 71573504, 4096 00020000:00200000:1.0:1789760779.672526:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760779.672549:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17473, 17473] 00000008:00010000:1.0:1789760779.672569:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760779.672575:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760779.672588:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71569408, 71573504) append 0 append lockpos 0 00000080:00200000:1.0:1789760779.672602:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71569408=0x4441000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760779.672707:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760779.672710:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71573504 00000020:00010000:1.0:1789760779.672719:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760779.672723:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.672726:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.672730:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760779.672733:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760779.672741:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760779.672747:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760779.672765:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760779.672890:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 71569408<>71573504 00000100:00100000:1.0:1789760779.672930:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760779.672950:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565466624:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.686725:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984781 gen 3 00000008:00010000:0.0:1789760779.686779:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760779.686789:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.686801:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.686818:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760779.686849:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565466624:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.687798:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565466752:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.697535:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984782 gen 3 00000008:00010000:3.0:1789760779.697607:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760779.697617:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.697628:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.697698:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760779.697730:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565466752:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.698752:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760779.698773:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565466880:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.708452:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984783 gen 3 00000008:00010000:3.0:1789760779.709468:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760779.709497:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.709518:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.709545:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760779.709790:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565466880:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.711169:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565467008:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.721179:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17476, 17476] 00000080:00200000:0.0:1789760779.721195:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.721201:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.721216:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71585792 i_blocks 139840 00000020:00200000:0.0:1789760779.721220:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.721231:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.721268:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71585792, count: 4096 00000080:00200000:0.0:1789760779.721275:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71585792, bytes: 4096 00000080:00200000:0.0:1789760779.721286:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.721294:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.721296:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71585792, 71589888) 0 0 00000080:00200000:0.0:1789760779.721301:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.721318:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17477, 17477] 00020000:00200000:0.0:1789760779.721326:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71585792 chunk: [71585792, 71589888) 71589888, 4096 00020000:00200000:0.0:1789760779.721332:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.721351:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17477, 17477] 00000008:00010000:0.0:1789760779.721369:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.721374:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.721386:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71585792, 71589888) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.721399:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71585792=0x4445000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.721458:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.721461:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71589888 00000020:00010000:0.0:1789760779.721469:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.721472:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.721475:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.721479:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.721482:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.721819:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.721827:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.721842:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.722013:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71585792<>71589888 00000100:00100000:1.0:1789760779.722074:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565467136:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.729143:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984785 gen 3 00000008:00010000:0.0:1789760779.729196:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.729205:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.729215:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.729297:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.729327:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565467136:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.730140:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565467264:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.739842:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984786 gen 3 00000008:00010000:3.0:1789760779.740365:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189a80 00000020:00200000:3.0:1789760779.740373:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.740384:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.740405:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189a80 00000100:00100000:3.0:1789760779.740435:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565467264:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.742268:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565467392:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.750524:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984787 gen 3 00000008:00010000:3.0:1789760779.750567:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760779.750574:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.750584:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.750601:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760779.750630:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565467392:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.752002:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760779.752022:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565467520:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.762522:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984788 gen 3 00000008:00010000:3.0:1789760779.762921:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760779.762930:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.762941:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.763088:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760779.763118:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565467520:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.765399:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565467648:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.779448:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17481, 17481] 00000080:00200000:0.0:1789760779.779475:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.779482:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.779508:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71606272 i_blocks 139880 00000020:00200000:0.0:1789760779.779514:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.779531:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.779571:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71606272, count: 4096 00000080:00200000:0.0:1789760779.779579:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71606272, bytes: 4096 00000080:00200000:0.0:1789760779.779593:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.779602:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.779604:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71606272, 71610368) 0 0 00000080:00200000:0.0:1789760779.779609:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.779626:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17482, 17482] 00020000:00200000:0.0:1789760779.779635:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71606272 chunk: [71606272, 71610368) 71610368, 4096 00020000:00200000:0.0:1789760779.779640:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.779697:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17482, 17482] 00000008:00010000:0.0:1789760779.779716:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.779722:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.779735:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71606272, 71610368) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.779751:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71606272=0x444a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.780055:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.780114:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71610368 00000020:00010000:0.0:1789760779.780125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.780129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.780132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.780135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.780138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.780147:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.780154:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.780177:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.780306:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71606272<>71610368 00000100:00100000:0.0:1789760779.780372:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760779.780397:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565467776:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.791197:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984790 gen 3 00000008:00010000:0.0:1789760779.791246:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.791259:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.791270:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.791338:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.791369:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565467776:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760779.791552:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17482, 17482] 00000080:00200000:1.0:1789760779.791627:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760779.791634:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760779.791744:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71610368 i_blocks 139888 00000020:00200000:1.0:1789760779.791750:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760779.791762:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760779.791798:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71610368, count: 4096 00000080:00200000:1.0:1789760779.791806:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71610368, bytes: 4096 00000080:00200000:1.0:1789760779.791815:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760779.791824:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760779.791826:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71610368, 71614464) 0 0 00000080:00200000:1.0:1789760779.791831:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760779.791847:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17483, 17483] 00020000:00200000:1.0:1789760779.791857:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71610368 chunk: [71610368, 71614464) 71614464, 4096 00020000:00200000:1.0:1789760779.791861:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760779.791882:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17483, 17483] 00000008:00010000:1.0:1789760779.791901:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760779.791907:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760779.791919:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71610368, 71614464) append 0 append lockpos 0 00000080:00200000:1.0:1789760779.791933:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71610368=0x444b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760779.791993:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760779.791996:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71614464 00000020:00010000:1.0:1789760779.792003:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760779.792006:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.792009:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760779.792013:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760779.792016:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760779.792022:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760779.792029:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760779.792048:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760779.792409:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 71610368<>71614464 00000100:00100000:1.0:1789760779.792450:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760779.792467:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565467904:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760779.803938:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760779.805416:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984791 gen 3 00000008:00010000:0.0:1789760779.805460:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760779.805468:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.805480:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.805517:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760779.805555:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565467904:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.806519:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565468032:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.819911:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17484, 17484] 00000080:00200000:0.0:1789760779.819928:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.819933:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.819947:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71618560 i_blocks 139904 00000020:00200000:0.0:1789760779.819950:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.819961:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.819996:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71618560, count: 4096 00000080:00200000:0.0:1789760779.820003:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71618560, bytes: 4096 00000080:00200000:0.0:1789760779.820014:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.820022:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.820025:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71618560, 71622656) 0 0 00000080:00200000:0.0:1789760779.820029:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.820047:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17485, 17485] 00020000:00200000:0.0:1789760779.820055:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71618560 chunk: [71618560, 71622656) 71622656, 4096 00020000:00200000:0.0:1789760779.820060:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.820080:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17485, 17485] 00000008:00010000:0.0:1789760779.820098:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.820104:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.820115:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71618560, 71622656) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.820129:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71618560=0x444d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.820187:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.820191:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71622656 00000020:00010000:0.0:1789760779.820199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.820203:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.820206:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.820210:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.820213:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.820221:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.820228:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.820246:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.820359:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71618560<>71622656 00000100:00100000:1.0:1789760779.820428:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565468160:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.833193:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984793 gen 3 00000008:00010000:0.0:1789760779.833312:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.833321:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.833334:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.833449:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.833489:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565468160:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760779.855245:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17486, 17486] 00000080:00200000:3.0:1789760779.855264:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760779.855271:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760779.855286:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71626752 i_blocks 139920 00000020:00200000:3.0:1789760779.855290:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760779.855301:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000100:00100000:3.0:1789760779.856268:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565468416:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.868536:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17487, 17487] 00000080:00200000:0.0:1789760779.868554:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.868627:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.868685:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71630848 i_blocks 139928 00000020:00200000:0.0:1789760779.868689:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.868700:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.868736:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71630848, count: 4096 00000080:00200000:0.0:1789760779.868744:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71630848, bytes: 4096 00000080:00200000:0.0:1789760779.868756:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.868764:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.868767:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71630848, 71634944) 0 0 00000080:00200000:0.0:1789760779.868771:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.868790:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17488, 17488] 00020000:00200000:0.0:1789760779.868799:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71630848 chunk: [71630848, 71634944) 71634944, 4096 00020000:00200000:0.0:1789760779.868805:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.868825:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17488, 17488] 00000008:00010000:0.0:1789760779.869008:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.869015:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.869030:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71630848, 71634944) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.869044:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71630848=0x4450000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.869102:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.869105:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71634944 00000020:00010000:0.0:1789760779.869114:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.869117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.869121:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.869125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.869127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.869136:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.869143:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.869163:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.869287:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71630848<>71634944 00000100:00100000:1.0:1789760779.869356:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565468544:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.879984:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984796 gen 3 00000008:00010000:0.0:1789760779.880036:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.880044:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.880055:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.880083:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.880113:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565468544:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760779.895075:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17489, 17489] 00000080:00200000:3.0:1789760779.895120:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760779.895126:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760779.896228:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565468800:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.906836:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17490, 17490] 00000080:00200000:0.0:1789760779.906965:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.906971:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.906987:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71643136 i_blocks 139952 00000020:00200000:0.0:1789760779.906991:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.907001:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.907037:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71643136, count: 4096 00000080:00200000:0.0:1789760779.907044:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71643136, bytes: 4096 00000080:00200000:0.0:1789760779.907057:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.907066:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.907068:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71643136, 71647232) 0 0 00000080:00200000:0.0:1789760779.907073:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.907090:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17491, 17491] 00020000:00200000:0.0:1789760779.907099:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71643136 chunk: [71643136, 71647232) 71647232, 4096 00020000:00200000:0.0:1789760779.907103:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.907125:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17491, 17491] 00000008:00010000:0.0:1789760779.907143:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.907149:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.907161:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71643136, 71647232) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.907175:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71643136=0x4453000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.907236:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.907240:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71647232 00000020:00010000:0.0:1789760779.907248:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.907315:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.907321:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.907325:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.907328:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.907338:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.907346:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.907369:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.907568:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71643136<>71647232 00000100:00100000:1.0:1789760779.907718:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565468928:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.919979:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984799 gen 3 00000008:00010000:0.0:1789760779.920581:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.920591:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.920603:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.920744:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.920776:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565468928:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.921842:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565469056:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.934042:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984800 gen 3 00000080:00200000:0.0:1789760779.934522:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17492, 17492] 00000080:00200000:0.0:1789760779.934539:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.934546:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.934562:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71651328 i_blocks 139968 00000020:00200000:0.0:1789760779.934566:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.934577:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.934615:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71651328, count: 4096 00000080:00200000:0.0:1789760779.934623:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71651328, bytes: 4096 00000080:00200000:0.0:1789760779.934635:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.934665:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.934668:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71651328, 71655424) 0 0 00000080:00200000:0.0:1789760779.934673:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.934690:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17493, 17493] 00020000:00200000:0.0:1789760779.934700:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71651328 chunk: [71651328, 71655424) 71655424, 4096 00020000:00200000:0.0:1789760779.934705:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.934726:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17493, 17493] 00000008:00010000:0.0:1789760779.934745:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.934751:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.934765:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71651328, 71655424) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.934839:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71651328=0x4455000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.934957:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.934960:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71655424 00000020:00010000:0.0:1789760779.934969:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.934972:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.934975:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.934979:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.934982:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.934990:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.934997:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.935014:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.935144:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71651328<>71655424 00000100:00100000:1.0:1789760779.935244:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565469184:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.946272:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984801 gen 3 00000008:00010000:0.0:1789760779.946324:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.946333:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.946344:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.946373:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.946403:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565469184:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760779.960631:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17494, 17494] 00000080:00200000:0.0:1789760779.960679:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760779.960685:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760779.960701:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71659520 i_blocks 139984 00000020:00200000:0.0:1789760779.960704:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760779.960715:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760779.960751:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71659520, count: 4096 00000080:00200000:0.0:1789760779.960758:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71659520, bytes: 4096 00000080:00200000:0.0:1789760779.960768:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760779.960777:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760779.960780:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71659520, 71663616) 0 0 00000080:00200000:0.0:1789760779.960784:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760779.960803:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17495, 17495] 00020000:00200000:0.0:1789760779.960814:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71659520 chunk: [71659520, 71663616) 71663616, 4096 00020000:00200000:0.0:1789760779.960819:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760779.960840:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17495, 17495] 00000008:00010000:0.0:1789760779.960860:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760779.960866:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760779.960878:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71659520, 71663616) append 0 append lockpos 0 00000080:00200000:0.0:1789760779.960892:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71659520=0x4457000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760779.960950:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760779.960953:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71663616 00000020:00010000:0.0:1789760779.960962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760779.960966:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.960969:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760779.961208:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760779.961212:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760779.961222:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760779.961230:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760779.961251:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760779.961377:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71659520<>71663616 00000100:00100000:0.0:1789760779.961431:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760779.961450:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565469440:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760779.971848:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984803 gen 3 00000008:00010000:0.0:1789760779.971896:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760779.971904:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760779.971914:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760779.972003:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760779.972031:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565469440:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.977183:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565469568:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.995107:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984804 gen 3 00000008:00010000:3.0:1789760779.995168:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760779.995177:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760779.995192:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760779.995222:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760779.995443:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565469568:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760779.997518:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760779.997549:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565469696:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.012517:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984805 gen 3 00000008:00010000:3.0:1789760780.012571:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760780.012580:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.012601:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.012624:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760780.012690:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565469696:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.014196:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565469824:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.036285:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984806 gen 3 00000008:00010000:3.0:1789760780.036336:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760780.036345:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.036356:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.036383:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760780.036410:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565469824:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.037545:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565469952:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760780.056663:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17499, 17499] 00000080:00200000:0.0:1789760780.056681:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.056687:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.056702:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71680000 i_blocks 140024 00000020:00200000:0.0:1789760780.056707:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.056717:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.056754:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71680000, count: 4096 00000080:00200000:0.0:1789760780.056762:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71680000, bytes: 4096 00000080:00200000:0.0:1789760780.056776:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.056786:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.056788:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71680000, 71684096) 0 0 00000080:00200000:0.0:1789760780.056793:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.056811:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17500, 17500] 00020000:00200000:0.0:1789760780.056821:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71680000 chunk: [71680000, 71684096) 71684096, 4096 00020000:00200000:0.0:1789760780.056826:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.056848:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17500, 17500] 00000008:00010000:0.0:1789760780.056867:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.056875:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.056888:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71680000, 71684096) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.056900:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71680000=0x445c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.056964:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.056967:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71684096 00000020:00010000:0.0:1789760780.056975:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.056978:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.056981:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.056985:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.056987:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.056996:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.057003:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.057025:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.057179:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71680000<>71684096 00000100:00100000:1.0:1789760780.057268:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565470080:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.069966:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984808 gen 3 00000008:00010000:0.0:1789760780.070017:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.070026:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.070036:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.070067:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.070097:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565470080:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.071039:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565470208:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.091725:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984809 gen 3 00000008:00010000:3.0:1789760780.091777:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760780.091785:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.091797:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.091879:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760780.091909:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565470208:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.094683:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565470336:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760780.110539:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17502, 17502] 00000080:00200000:3.0:1789760780.110592:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.110599:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.110615:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71692288 i_blocks 140048 00000020:00200000:3.0:1789760780.110619:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.110628:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200000:3.0:1789760780.127491:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17503, 17503] 00000080:00200000:3.0:1789760780.127546:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.127553:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.127567:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71696384 i_blocks 140056 00000020:00200000:3.0:1789760780.127571:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.127591:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760780.127640:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71696384, count: 4096 00000080:00200000:3.0:1789760780.127701:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71696384, bytes: 4096 00000080:00200000:3.0:1789760780.127711:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760780.127721:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760780.127723:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71696384, 71700480) 0 0 00000080:00200000:3.0:1789760780.127728:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760780.127747:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17504, 17504] 00020000:00200000:3.0:1789760780.127757:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71696384 chunk: [71696384, 71700480) 71700480, 4096 00020000:00200000:3.0:1789760780.127761:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760780.127781:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17504, 17504] 00000008:00010000:3.0:1789760780.127804:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760780.127810:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760780.127822:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71696384, 71700480) append 0 append lockpos 0 00000080:00200000:3.0:1789760780.127836:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71696384=0x4460000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760780.127896:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760780.127899:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71700480 00000020:00010000:3.0:1789760780.127907:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760780.127911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.127914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.127918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760780.127920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760780.127929:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760780.127935:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760780.127953:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760780.128083:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66d80 - object 0x240000400:5 offset 71696384<>71700480 00000080:00200000:3.0:1789760780.143465:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17504, 17504] 00000080:00200000:3.0:1789760780.143482:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.143489:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.143514:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71700480 i_blocks 140064 00000020:00200000:3.0:1789760780.143518:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.143529:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760780.143564:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71700480, count: 4096 00000080:00200000:3.0:1789760780.143572:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71700480, bytes: 4096 00000080:00200000:3.0:1789760780.143591:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760780.143601:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760780.143603:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71700480, 71704576) 0 0 00000080:00200000:3.0:1789760780.143608:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760780.143625:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17505, 17505] 00020000:00200000:3.0:1789760780.143634:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71700480 chunk: [71700480, 71704576) 71704576, 4096 00020000:00200000:3.0:1789760780.143639:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760780.143680:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17505, 17505] 00000008:00010000:3.0:1789760780.143699:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760780.143706:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760780.143718:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71700480, 71704576) append 0 append lockpos 0 00000080:00200000:3.0:1789760780.143732:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71700480=0x4461000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760780.143786:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760780.143789:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71704576 00000020:00010000:3.0:1789760780.143797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760780.143801:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.143804:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.143807:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760780.143810:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760780.143819:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760780.143827:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760780.143847:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760780.143973:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 71700480<>71704576 00000080:00200000:3.0:1789760780.158640:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17505, 17505] 00000080:00200000:3.0:1789760780.158675:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.158681:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760780.159307:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565470848:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760780.171147:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17506, 17506] 00000080:00200000:0.0:1789760780.171162:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.171168:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.171183:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71708672 i_blocks 140080 00000020:00200000:0.0:1789760780.171187:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.171198:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.171248:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71708672, count: 4096 00000080:00200000:0.0:1789760780.171255:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71708672, bytes: 4096 00000080:00200000:0.0:1789760780.171265:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.171275:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.171277:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71708672, 71712768) 0 0 00000080:00200000:0.0:1789760780.171282:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.171299:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17507, 17507] 00020000:00200000:0.0:1789760780.171309:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71708672 chunk: [71708672, 71712768) 71712768, 4096 00020000:00200000:0.0:1789760780.171314:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.171333:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17507, 17507] 00000008:00010000:0.0:1789760780.171352:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.171358:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.171370:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71708672, 71712768) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.171384:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71708672=0x4463000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.171442:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.171451:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71712768 00000020:00010000:0.0:1789760780.171460:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.171464:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.171467:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.171470:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.171473:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.171481:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.171488:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.171513:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.171637:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71708672<>71712768 00000100:00100000:0.0:1789760780.171712:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760780.171728:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565470976:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.189621:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984815 gen 3 00000008:00010000:0.0:1789760780.189847:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.189856:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.189868:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.189957:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.189987:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565470976:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760780.209566:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17508, 17508] 00000080:00200000:3.0:1789760780.209583:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.209590:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.209607:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71716864 i_blocks 140096 00000020:00200000:3.0:1789760780.209611:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.209622:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760780.209715:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71716864, count: 4096 00000080:00200000:3.0:1789760780.209723:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71716864, bytes: 4096 00000080:00200000:3.0:1789760780.209736:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760780.209744:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760780.209747:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71716864, 71720960) 0 0 00000080:00200000:3.0:1789760780.209751:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760780.209767:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17509, 17509] 00020000:00200000:3.0:1789760780.209777:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71716864 chunk: [71716864, 71720960) 71720960, 4096 00020000:00200000:3.0:1789760780.209782:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760780.209802:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17509, 17509] 00000008:00010000:3.0:1789760780.210364:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760780.210372:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760780.210388:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71716864, 71720960) append 0 append lockpos 0 00000080:00200000:3.0:1789760780.210403:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71716864=0x4465000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760780.210546:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760780.210549:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71720960 00000020:00010000:3.0:1789760780.210558:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760780.210562:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.210565:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.210569:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760780.210571:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760780.210582:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760780.210589:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760780.210617:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760780.211034:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 71716864<>71720960 00000080:00200000:0.0:1789760780.233272:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17509, 17509] 00000080:00200000:0.0:1789760780.233290:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.233296:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.233311:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71720960 i_blocks 140104 00000020:00200000:0.0:1789760780.233315:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.233325:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.233357:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71720960, count: 4096 00000080:00200000:0.0:1789760780.233366:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71720960, bytes: 4096 00000080:00200000:0.0:1789760780.233378:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.233386:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.233389:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71720960, 71725056) 0 0 00000080:00200000:0.0:1789760780.233393:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.233409:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17510, 17510] 00020000:00200000:0.0:1789760780.233418:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71720960 chunk: [71720960, 71725056) 71725056, 4096 00020000:00200000:0.0:1789760780.233422:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.233511:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17510, 17510] 00000008:00010000:0.0:1789760780.233528:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.233534:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.233545:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71720960, 71725056) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.233558:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71720960=0x4466000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.233611:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.233614:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71725056 00000020:00010000:0.0:1789760780.233621:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.233625:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.233627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.233631:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.233635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.233659:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.233665:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.233678:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.233791:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71720960<>71725056 00000100:00100000:1.0:1789760780.233877:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565471360:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.251880:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984818 gen 3 00000008:00010000:0.0:1789760780.251929:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.251937:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.252021:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.252045:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.252076:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565471360:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.252855:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565471488:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.266072:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17511, 17511] 00000080:00200000:1.0:1789760780.266088:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.266094:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.266108:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71729152 i_blocks 140120 00000020:00200000:1.0:1789760780.266112:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.266121:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.266157:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71729152, count: 4096 00000080:00200000:1.0:1789760780.266165:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71729152, bytes: 4096 00000080:00200000:1.0:1789760780.266175:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.266183:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.266186:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71729152, 71733248) 0 0 00000080:00200000:1.0:1789760780.266190:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.266206:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17512, 17512] 00020000:00200000:1.0:1789760780.266215:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71729152 chunk: [71729152, 71733248) 71733248, 4096 00020000:00200000:1.0:1789760780.266220:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.266241:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17512, 17512] 00000008:00010000:1.0:1789760780.266261:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.266268:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.266278:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71729152, 71733248) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.266292:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71729152=0x4468000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.266349:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.266352:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71733248 00000020:00010000:1.0:1789760780.266361:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.266364:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.266367:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.266370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.266373:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.266380:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.266387:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.266403:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760780.266554:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 71729152<>71733248 00000100:00100000:0.0:1789760780.266629:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565471616:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.279796:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984820 gen 3 00000008:00010000:0.0:1789760780.279846:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760780.279855:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.279865:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.279890:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760780.279921:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565471616:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.280045:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17512, 17512] 00000080:00200000:1.0:1789760780.280062:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.280068:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.280085:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71733248 i_blocks 140128 00000020:00200000:1.0:1789760780.280088:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.280099:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.280134:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71733248, count: 4096 00000080:00200000:1.0:1789760780.280142:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71733248, bytes: 4096 00000080:00200000:1.0:1789760780.280154:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.280163:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.280165:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71733248, 71737344) 0 0 00000080:00200000:1.0:1789760780.280170:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.280187:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17513, 17513] 00020000:00200000:1.0:1789760780.280196:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71733248 chunk: [71733248, 71737344) 71737344, 4096 00020000:00200000:1.0:1789760780.280201:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.280220:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17513, 17513] 00000008:00010000:1.0:1789760780.280242:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.280248:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.280261:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71733248, 71737344) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.280276:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71733248=0x4469000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.280337:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.280340:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71737344 00000020:00010000:1.0:1789760780.280349:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.280353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.280356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.280360:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.280363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.280371:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.280378:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.280395:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760780.280829:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 71733248<>71737344 00000100:00100000:0.0:1789760780.281358:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565471744:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.291800:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984821 gen 3 00000008:00010000:0.0:1789760780.291853:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760780.291861:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.291873:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.291895:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760780.291926:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565471744:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.292308:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17513, 17513] 00000080:00200000:1.0:1789760780.292325:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.292332:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.292347:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71737344 i_blocks 140136 00000020:00200000:1.0:1789760780.292351:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.292362:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.292398:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71737344, count: 4096 00000080:00200000:1.0:1789760780.292406:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71737344, bytes: 4096 00000080:00200000:1.0:1789760780.292416:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.292425:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.292427:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71737344, 71741440) 0 0 00000080:00200000:1.0:1789760780.292432:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.292449:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17514, 17514] 00020000:00200000:1.0:1789760780.292827:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71737344 chunk: [71737344, 71741440) 71741440, 4096 00020000:00200000:1.0:1789760780.292834:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.292857:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17514, 17514] 00000008:00010000:1.0:1789760780.292876:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.292882:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.292895:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71737344, 71741440) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.292909:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71737344=0x446a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.292980:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.292982:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71741440 00000020:00010000:1.0:1789760780.292990:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.292994:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.292997:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.293000:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.293002:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.293010:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.293017:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.293037:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760780.293162:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 71737344<>71741440 00000100:00100000:0.0:1789760780.293302:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565471872:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.303417:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984822 gen 3 00000008:00010000:0.0:1789760780.303633:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760780.303641:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.303946:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.304007:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760780.304040:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565471872:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.304737:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17514, 17514] 00000080:00200000:1.0:1789760780.304756:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.304762:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.304946:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71741440 i_blocks 140144 00000020:00200000:1.0:1789760780.304951:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.304964:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.305001:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71741440, count: 4096 00000080:00200000:1.0:1789760780.305009:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71741440, bytes: 4096 00000080:00200000:1.0:1789760780.305022:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.305032:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.305034:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71741440, 71745536) 0 0 00000080:00200000:1.0:1789760780.305039:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.305185:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17515, 17515] 00020000:00200000:1.0:1789760780.305195:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71741440 chunk: [71741440, 71745536) 71745536, 4096 00020000:00200000:1.0:1789760780.305200:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.305223:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17515, 17515] 00000008:00010000:1.0:1789760780.305243:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.305249:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.305260:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71741440, 71745536) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.305275:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71741440=0x446b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.305347:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.305350:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71745536 00000020:00010000:1.0:1789760780.305359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.305362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.305366:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.305370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.305372:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.305380:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.305386:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.305413:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760780.306010:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 71741440<>71745536 00000100:00100000:0.0:1789760780.306075:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565472000:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.317617:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984823 gen 3 00000008:00010000:0.0:1789760780.317821:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760780.317830:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.317843:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.317871:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760780.317899:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565472000:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.318683:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565472128:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.338621:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984824 gen 3 00000008:00010000:3.0:1789760780.339619:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760780.339630:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.339790:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.339865:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760780.339899:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565472128:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760780.341785:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17516, 17516] 00000080:00200000:0.0:1789760780.341811:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.341818:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.341855:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71749632 i_blocks 140160 00000020:00200000:0.0:1789760780.341859:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.341870:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.341904:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71749632, count: 4096 00000080:00200000:0.0:1789760780.341912:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71749632, bytes: 4096 00000080:00200000:0.0:1789760780.341927:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.341937:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.341939:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71749632, 71753728) 0 0 00000080:00200000:0.0:1789760780.341944:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.341960:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17517, 17517] 00020000:00200000:0.0:1789760780.341969:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71749632 chunk: [71749632, 71753728) 71753728, 4096 00020000:00200000:0.0:1789760780.341974:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.341998:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17517, 17517] 00000008:00010000:0.0:1789760780.342016:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.342022:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.342035:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71749632, 71753728) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.342048:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71749632=0x446d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.342112:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.342116:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71753728 00000020:00010000:0.0:1789760780.342124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.342128:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.342131:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.342135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.342138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.342148:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.342156:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.342171:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.342894:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71749632<>71753728 00000100:00100000:0.0:1789760780.342950:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760780.342973:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565472256:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.356758:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984825 gen 3 00000008:00010000:0.0:1789760780.356811:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.356820:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.356831:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.356973:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.357005:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565472256:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760780.357118:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17517, 17517] 00000080:00200000:3.0:1789760780.357134:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.357141:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.357157:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71753728 i_blocks 140168 00000020:00200000:3.0:1789760780.357160:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.357172:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760780.357208:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71753728, count: 4096 00000080:00200000:3.0:1789760780.357216:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71753728, bytes: 4096 00000080:00200000:3.0:1789760780.357227:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760780.357237:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760780.357239:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71753728, 71757824) 0 0 00000080:00200000:3.0:1789760780.357244:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760780.357262:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17518, 17518] 00020000:00200000:3.0:1789760780.357271:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71753728 chunk: [71753728, 71757824) 71757824, 4096 00020000:00200000:3.0:1789760780.357276:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760780.357296:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17518, 17518] 00000008:00010000:3.0:1789760780.357314:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760780.357319:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760780.357331:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71753728, 71757824) append 0 append lockpos 0 00000080:00200000:3.0:1789760780.357345:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71753728=0x446e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760780.357407:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760780.357410:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71757824 00000020:00010000:3.0:1789760780.357419:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760780.357423:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.357427:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.357431:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760780.357434:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760780.357442:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760780.357449:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760780.358053:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760780.358191:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 71753728<>71757824 00000080:00200000:3.0:1789760780.369381:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17518, 17518] 00000080:00200000:3.0:1789760780.369397:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.369402:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.369418:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71757824 i_blocks 140176 00000020:00200000:3.0:1789760780.369421:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.369432:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760780.369475:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71757824, count: 4096 00000080:00200000:3.0:1789760780.369482:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71757824, bytes: 4096 00000080:00200000:3.0:1789760780.369501:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760780.369511:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760780.369513:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71757824, 71761920) 0 0 00000080:00200000:3.0:1789760780.369518:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760780.369536:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17519, 17519] 00020000:00200000:3.0:1789760780.369545:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71757824 chunk: [71757824, 71761920) 71761920, 4096 00020000:00200000:3.0:1789760780.369550:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760780.369568:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17519, 17519] 00000008:00010000:3.0:1789760780.369584:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760780.369590:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760780.369602:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71757824, 71761920) append 0 append lockpos 0 00000080:00200000:3.0:1789760780.369616:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71757824=0x446f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760780.369974:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760780.369977:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71761920 00000020:00010000:3.0:1789760780.369986:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760780.369990:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.369994:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.369998:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760780.370001:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760780.370010:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760780.370017:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760780.370034:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760780.370155:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 71757824<>71761920 00000080:00200000:3.0:1789760780.384073:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17519, 17519] 00000080:00200000:3.0:1789760780.384091:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.384096:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.384111:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71761920 i_blocks 140184 00000020:00200000:3.0:1789760780.384115:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.384125:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760780.384160:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71761920, count: 4096 00000080:00200000:3.0:1789760780.384167:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71761920, bytes: 4096 00000080:00200000:3.0:1789760780.384179:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760780.384187:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760780.384190:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71761920, 71766016) 0 0 00000080:00200000:3.0:1789760780.384195:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760780.384212:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17520, 17520] 00020000:00200000:3.0:1789760780.384221:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71761920 chunk: [71761920, 71766016) 71766016, 4096 00020000:00200000:3.0:1789760780.384226:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760780.384244:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17520, 17520] 00000008:00010000:3.0:1789760780.384261:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760780.384267:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760780.384455:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71761920, 71766016) append 0 append lockpos 0 00000080:00200000:3.0:1789760780.384470:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71761920=0x4470000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760780.384538:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760780.384542:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71766016 00000020:00010000:3.0:1789760780.384620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760780.384624:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.384628:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.384632:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760780.384635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760780.385812:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760780.385826:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760780.385857:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760780.385981:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 71761920<>71766016 00000080:00200000:3.0:1789760780.402963:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17520, 17520] 00000080:00200000:3.0:1789760780.402973:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.402977:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.402987:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71766016 i_blocks 140192 00000020:00200000:3.0:1789760780.402990:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.403057:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760780.403079:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71766016, count: 4096 00000080:00200000:3.0:1789760780.403084:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71766016, bytes: 4096 00000080:00200000:3.0:1789760780.403093:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760780.403099:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760780.403101:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71766016, 71770112) 0 0 00000080:00200000:3.0:1789760780.403104:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760780.403115:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17521, 17521] 00020000:00200000:3.0:1789760780.403120:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71766016 chunk: [71766016, 71770112) 71770112, 4096 00020000:00200000:3.0:1789760780.403124:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760780.403137:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17521, 17521] 00000008:00010000:3.0:1789760780.403157:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760780.403161:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760780.403169:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71766016, 71770112) append 0 append lockpos 0 00000080:00200000:3.0:1789760780.403177:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71766016=0x4471000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760780.403287:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760780.403289:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71770112 00000020:00010000:3.0:1789760780.403295:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760780.403298:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.403300:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.403302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760780.403304:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760780.403309:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760780.403313:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760780.403323:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760780.403406:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 71766016<>71770112 00000080:00200000:3.0:1789760780.419444:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17521, 17521] 00000080:00200000:3.0:1789760780.419464:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.419469:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.419551:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71770112 i_blocks 140200 00000020:00200000:3.0:1789760780.419557:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.419570:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760780.419607:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71770112, count: 4096 00000080:00200000:3.0:1789760780.419615:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71770112, bytes: 4096 00000080:00200000:3.0:1789760780.419624:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760780.419630:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760780.419632:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71770112, 71774208) 0 0 00000080:00200000:3.0:1789760780.419638:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760780.419680:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17522, 17522] 00020000:00200000:3.0:1789760780.419943:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71770112 chunk: [71770112, 71774208) 71774208, 4096 00020000:00200000:3.0:1789760780.419948:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760780.419969:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17522, 17522] 00000008:00010000:3.0:1789760780.419988:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760780.419994:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760780.420005:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71770112, 71774208) append 0 append lockpos 0 00000080:00200000:3.0:1789760780.420018:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71770112=0x4472000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760780.420080:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760780.420083:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71774208 00000020:00010000:3.0:1789760780.420091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760780.420094:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.420098:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.420102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760780.420104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760780.420114:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760780.420120:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760780.420146:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760780.420344:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66d80 - object 0x240000400:5 offset 71770112<>71774208 00000080:00200000:3.0:1789760780.435957:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17522, 17522] 00000080:00200000:3.0:1789760780.435974:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.435984:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.436001:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71774208 i_blocks 140208 00000020:00200000:3.0:1789760780.436004:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.436014:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760780.436049:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71774208, count: 4096 00000080:00200000:3.0:1789760780.436057:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71774208, bytes: 4096 00000080:00200000:3.0:1789760780.436067:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760780.436076:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760780.436079:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71774208, 71778304) 0 0 00000080:00200000:3.0:1789760780.436083:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760780.436100:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17523, 17523] 00020000:00200000:3.0:1789760780.436109:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71774208 chunk: [71774208, 71778304) 71778304, 4096 00020000:00200000:3.0:1789760780.436113:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760780.436135:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17523, 17523] 00000008:00010000:3.0:1789760780.436154:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760780.436160:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760780.436173:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71774208, 71778304) append 0 append lockpos 0 00000080:00200000:3.0:1789760780.436187:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71774208=0x4473000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760780.436253:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760780.436256:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71778304 00000020:00010000:3.0:1789760780.436266:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760780.436269:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.436273:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.436277:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760780.436280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760780.436291:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760780.436298:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760780.436316:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760780.437022:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 71774208<>71778304 00000100:00100000:3.0:1789760780.437085:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565473024:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760780.452994:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17523, 17523] 00000080:00200000:0.0:1789760780.453010:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.453016:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.453031:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71778304 i_blocks 140216 00000020:00200000:0.0:1789760780.453035:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.453047:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.453079:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71778304, count: 4096 00000080:00200000:0.0:1789760780.453087:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71778304, bytes: 4096 00000080:00200000:0.0:1789760780.453097:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.453105:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.453107:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71778304, 71782400) 0 0 00000080:00200000:0.0:1789760780.453112:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.453128:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17524, 17524] 00020000:00200000:0.0:1789760780.453137:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71778304 chunk: [71778304, 71782400) 71782400, 4096 00020000:00200000:0.0:1789760780.453141:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.453160:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17524, 17524] 00000008:00010000:0.0:1789760780.453177:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.453183:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.453194:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71778304, 71782400) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.453206:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71778304=0x4474000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.453337:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.453340:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71782400 00000020:00010000:0.0:1789760780.453349:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.453353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.453356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.453359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.453362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.453371:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.453377:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.453395:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.453527:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71778304<>71782400 00000100:00100000:1.0:1789760780.453596:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565473152:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.465998:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984832 gen 3 00000008:00010000:0.0:1789760780.466046:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.466055:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.466065:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.466090:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.466120:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565473152:192.168.203.147@tcp:4:dd.0 00000020:00010000:2.0:1789760780.466569:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.466573:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.466577:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.466580:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.466583:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.466589:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.466596:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.466612:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f8441893c0 00000008:00100000:2.0:1789760780.467002:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 71782400<>71786496 00000100:00100000:2.0:1789760780.467053:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760780.467073:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565473280:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760780.477513:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984833 gen 3 00000008:00010000:2.0:1789760780.477567:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:2.0:1789760780.477576:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760780.477587:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760780.477614:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000080:00200000:0.0:1789760780.477774:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17525, 17525] 00000080:00200000:0.0:1789760780.477794:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.477800:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.477816:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71786496 i_blocks 140232 00000020:00200000:0.0:1789760780.477820:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.477830:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000100:00100000:2.0:1789760780.477926:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565473280:192.168.203.147@tcp:4:dd.0 00000080:00200008:0.0:1789760780.478016:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71786496, count: 4096 00000080:00200000:0.0:1789760780.478025:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71786496, bytes: 4096 00000080:00200000:0.0:1789760780.478036:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.478045:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.478048:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71786496, 71790592) 0 0 00000080:00200000:0.0:1789760780.478053:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.478070:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17526, 17526] 00020000:00200000:0.0:1789760780.478081:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71786496 chunk: [71786496, 71790592) 71790592, 4096 00020000:00200000:0.0:1789760780.478086:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.478105:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17526, 17526] 00000008:00010000:0.0:1789760780.478124:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.478130:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.478142:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71786496, 71790592) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.478155:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71786496=0x4476000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.478218:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.478221:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71790592 00000020:00010000:0.0:1789760780.478230:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.478233:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.478237:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.478240:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.478243:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.478250:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.478257:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.478274:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.478394:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71786496<>71790592 00000100:00100000:1.0:1789760780.478462:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565473408:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.490619:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984834 gen 3 00000008:00010000:0.0:1789760780.490706:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.490714:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.490725:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.490750:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.490778:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565473408:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.491003:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17526, 17526] 00000080:00200000:2.0:1789760780.491019:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.491024:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.491040:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71790592 i_blocks 140240 00000020:00200000:2.0:1789760780.491043:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.491055:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.491087:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71790592, count: 4096 00000080:00200000:2.0:1789760780.491095:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71790592, bytes: 4096 00000080:00200000:2.0:1789760780.491105:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.491114:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.491116:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71790592, 71794688) 0 0 00000080:00200000:2.0:1789760780.491120:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.491136:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17527, 17527] 00020000:00200000:2.0:1789760780.491144:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71790592 chunk: [71790592, 71794688) 71794688, 4096 00020000:00200000:2.0:1789760780.491149:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.491167:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17527, 17527] 00000008:00010000:2.0:1789760780.492815:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.492828:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.492847:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71790592, 71794688) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.492864:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71790592=0x4477000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.492942:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.492946:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71794688 00000020:00010000:2.0:1789760780.492954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.492958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.492960:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.492963:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.492966:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.492976:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.492985:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.493012:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f8441893c0 00000008:00100000:2.0:1789760780.493132:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 71790592<>71794688 00000100:00100000:3.0:1789760780.493332:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565473536:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.507174:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984835 gen 3 00000008:00010000:3.0:1789760780.507225:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760780.507235:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.507246:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.507272:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760780.507304:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565473536:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.507481:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17527, 17527] 00000080:00200000:2.0:1789760780.507505:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.507511:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.507526:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71794688 i_blocks 140248 00000020:00200000:2.0:1789760780.507529:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.507539:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.507569:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71794688, count: 4096 00000080:00200000:2.0:1789760780.507577:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71794688, bytes: 4096 00000080:00200000:2.0:1789760780.507590:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.507599:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.507602:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71794688, 71798784) 0 0 00000080:00200000:2.0:1789760780.507606:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.507621:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17528, 17528] 00020000:00200000:2.0:1789760780.507632:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71794688 chunk: [71794688, 71798784) 71798784, 4096 00020000:00200000:2.0:1789760780.507637:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.507679:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17528, 17528] 00000008:00010000:2.0:1789760780.507699:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.507706:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.507718:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71794688, 71798784) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.507732:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71794688=0x4478000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.507790:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.507793:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71798784 00000020:00010000:2.0:1789760780.507802:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.507805:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.507809:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.507812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.507815:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.507823:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.507830:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.507846:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760780.507969:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 71794688<>71798784 00000100:00100000:3.0:1789760780.508125:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760780.508143:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565473664:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.518101:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984836 gen 3 00000008:00010000:3.0:1789760780.518153:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760780.518161:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.518171:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.518194:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760780.518223:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565473664:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.518342:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17528, 17528] 00000080:00200000:2.0:1789760780.518359:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.518365:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.518380:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71798784 i_blocks 140256 00000020:00200000:2.0:1789760780.518383:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.518392:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.518424:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71798784, count: 4096 00000080:00200000:2.0:1789760780.518500:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71798784, bytes: 4096 00000080:00200000:2.0:1789760780.518510:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.518519:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.518522:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71798784, 71802880) 0 0 00000080:00200000:2.0:1789760780.518526:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.518544:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17529, 17529] 00020000:00200000:2.0:1789760780.518553:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71798784 chunk: [71798784, 71802880) 71802880, 4096 00020000:00200000:2.0:1789760780.518558:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.518579:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17529, 17529] 00000008:00010000:2.0:1789760780.518596:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.518603:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.518615:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71798784, 71802880) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.518628:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71798784=0x4479000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.518720:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.518723:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71802880 00000020:00010000:2.0:1789760780.518732:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.518735:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.518738:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.518741:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.518744:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.518752:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.518759:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.518775:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760780.518889:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 71798784<>71802880 00000100:00100000:3.0:1789760780.518960:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565473792:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760780.527573:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984837 gen 3 00000008:00010000:2.0:1789760780.527713:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760780.527722:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760780.527733:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760780.527809:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760780.527840:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565473792:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760780.527919:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17529, 17529] 00000080:00200000:0.0:1789760780.527935:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.527942:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.527958:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71802880 i_blocks 140264 00000020:00200000:0.0:1789760780.527962:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.527972:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.528007:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71802880, count: 4096 00000080:00200000:0.0:1789760780.528015:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71802880, bytes: 4096 00000080:00200000:0.0:1789760780.528025:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.528034:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.528037:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71802880, 71806976) 0 0 00000080:00200000:0.0:1789760780.528041:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.528059:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17530, 17530] 00020000:00200000:0.0:1789760780.528068:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71802880 chunk: [71802880, 71806976) 71806976, 4096 00020000:00200000:0.0:1789760780.528072:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.528091:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17530, 17530] 00000008:00010000:0.0:1789760780.528110:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.528117:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.528128:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71802880, 71806976) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.528142:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71802880=0x447a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.528260:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.528263:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71806976 00000020:00010000:0.0:1789760780.528272:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.528275:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.528279:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.528282:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.528285:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.528293:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.528300:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.528315:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.528504:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71802880<>71806976 00000100:00100000:1.0:1789760780.528725:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565473920:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.542222:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984838 gen 3 00000008:00010000:0.0:1789760780.542274:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.542283:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.542294:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.542318:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.542349:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565473920:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.542562:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17530, 17530] 00000080:00200000:2.0:1789760780.543306:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.543315:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.543334:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71806976 i_blocks 140272 00000020:00200000:2.0:1789760780.543338:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.543353:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.543390:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71806976, count: 4096 00000080:00200000:2.0:1789760780.543398:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71806976, bytes: 4096 00000080:00200000:2.0:1789760780.543998:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.544010:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.544012:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71806976, 71811072) 0 0 00000080:00200000:2.0:1789760780.544017:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.544035:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17531, 17531] 00020000:00200000:2.0:1789760780.544045:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71806976 chunk: [71806976, 71811072) 71811072, 4096 00020000:00200000:2.0:1789760780.544050:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.544070:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17531, 17531] 00000008:00010000:2.0:1789760780.544096:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.544102:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.544113:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71806976, 71811072) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.544126:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71806976=0x447b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.544196:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.544198:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71811072 00000020:00010000:2.0:1789760780.544206:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.544209:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.544212:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.544215:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.544218:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.544227:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.544234:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.544258:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760780.544379:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 71806976<>71811072 00000100:00100000:2.0:1789760780.544429:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760780.544452:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565474048:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760780.556856:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984839 gen 3 00000008:00010000:2.0:1789760780.556906:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760780.556916:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760780.556928:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760780.556958:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760780.556989:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565474048:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760780.557320:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17531, 17531] 00000080:00200000:3.0:1789760780.557337:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760780.557343:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760780.557360:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71811072 i_blocks 140280 00000020:00200000:3.0:1789760780.557364:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760780.557375:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760780.557420:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71811072, count: 4096 00000080:00200000:3.0:1789760780.557429:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71811072, bytes: 4096 00000080:00200000:3.0:1789760780.557440:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760780.557450:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760780.557452:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71811072, 71815168) 0 0 00000080:00200000:3.0:1789760780.557457:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760780.557474:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17532, 17532] 00020000:00200000:3.0:1789760780.557483:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71811072 chunk: [71811072, 71815168) 71815168, 4096 00020000:00200000:3.0:1789760780.557489:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760780.557527:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17532, 17532] 00000008:00010000:3.0:1789760780.557545:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760780.557550:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760780.557562:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71811072, 71815168) append 0 append lockpos 0 00000080:00200000:3.0:1789760780.557575:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71811072=0x447c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760780.558055:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760780.558059:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71815168 00000020:00010000:3.0:1789760780.558069:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760780.558073:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.558076:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760780.558080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760780.558083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760780.558092:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760780.558100:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760780.558121:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760780.558799:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 71811072<>71815168 00000100:00100000:2.0:1789760780.558909:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565474176:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760780.571046:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984840 gen 3 00000008:00010000:2.0:1789760780.571093:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760780.571101:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760780.571113:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760780.571273:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760780.571316:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565474176:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760780.571419:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17532, 17532] 00000080:00200000:0.0:1789760780.571437:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.571444:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.571461:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71815168 i_blocks 140288 00000020:00200000:0.0:1789760780.571466:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.571477:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.571530:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71815168, count: 4096 00000080:00200000:0.0:1789760780.571539:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71815168, bytes: 4096 00000080:00200000:0.0:1789760780.571551:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.571560:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.571562:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71815168, 71819264) 0 0 00000080:00200000:0.0:1789760780.571567:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.571755:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17533, 17533] 00020000:00200000:0.0:1789760780.572413:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71815168 chunk: [71815168, 71819264) 71819264, 4096 00020000:00200000:0.0:1789760780.572421:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.572446:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17533, 17533] 00000008:00010000:0.0:1789760780.572469:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.572484:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.572505:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71815168, 71819264) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.572519:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71815168=0x447d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.572602:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.572605:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71819264 00000020:00010000:0.0:1789760780.572613:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.572617:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.572620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.572624:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.572627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.572635:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.572675:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.572701:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.572837:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71815168<>71819264 00000100:00100000:0.0:1789760780.572894:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760780.572915:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565474304:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.585242:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984841 gen 3 00000008:00010000:0.0:1789760780.585946:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.585957:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.585971:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.586000:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.586032:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565474304:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.586715:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17533, 17533] 00000080:00200000:2.0:1789760780.586735:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.586740:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.586756:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71819264 i_blocks 140296 00000020:00200000:2.0:1789760780.586760:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.586774:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.586813:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71819264, count: 4096 00000080:00200000:2.0:1789760780.586820:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71819264, bytes: 4096 00000080:00200000:2.0:1789760780.586835:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.586845:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.586847:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71819264, 71823360) 0 0 00000080:00200000:2.0:1789760780.586852:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.586870:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17534, 17534] 00020000:00200000:2.0:1789760780.586880:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71819264 chunk: [71819264, 71823360) 71823360, 4096 00020000:00200000:2.0:1789760780.586885:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.586905:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17534, 17534] 00000008:00010000:2.0:1789760780.586924:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.586931:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.586943:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71819264, 71823360) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.586962:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71819264=0x447e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.587024:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.587027:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71823360 00000020:00010000:2.0:1789760780.587035:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.587039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.587042:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.587045:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.587048:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.587055:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.587063:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.587079:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760780.587204:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 71819264<>71823360 00000100:00100000:3.0:1789760780.588793:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565474432:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.600799:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984842 gen 3 00000008:00010000:3.0:1789760780.600852:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760780.600861:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.600873:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.600903:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760780.600935:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565474432:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.601058:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17534, 17534] 00000080:00200000:2.0:1789760780.601075:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.601082:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.601098:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71823360 i_blocks 140304 00000020:00200000:2.0:1789760780.601102:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.601112:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.601801:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71823360, count: 4096 00000080:00200000:2.0:1789760780.601814:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71823360, bytes: 4096 00000080:00200000:2.0:1789760780.601829:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.601839:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.601841:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71823360, 71827456) 0 0 00000080:00200000:2.0:1789760780.601846:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.601866:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17535, 17535] 00020000:00200000:2.0:1789760780.601876:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71823360 chunk: [71823360, 71827456) 71827456, 4096 00020000:00200000:2.0:1789760780.601881:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.601905:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17535, 17535] 00000008:00010000:2.0:1789760780.601926:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.601932:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.601945:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71823360, 71827456) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.601961:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71823360=0x447f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.602029:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.602032:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71827456 00000020:00010000:2.0:1789760780.602041:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.602045:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.602048:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.602051:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.602054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.602064:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.602072:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.602099:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760780.603473:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 71823360<>71827456 00000100:00100000:3.0:1789760780.604278:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565474560:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.624706:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984843 gen 3 00000008:00010000:3.0:1789760780.624758:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760780.624766:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.624777:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.624897:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760780.624930:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565474560:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.625063:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17535, 17535] 00000080:00200000:2.0:1789760780.625081:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.625086:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.625103:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71827456 i_blocks 140312 00000020:00200000:2.0:1789760780.625107:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.625117:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.625221:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71827456, count: 4096 00000080:00200000:2.0:1789760780.625230:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71827456, bytes: 4096 00000080:00200000:2.0:1789760780.625240:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.625249:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.625251:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71827456, 71831552) 0 0 00000080:00200000:2.0:1789760780.625256:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.625272:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17536, 17536] 00020000:00200000:2.0:1789760780.625391:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71827456 chunk: [71827456, 71831552) 71831552, 4096 00020000:00200000:2.0:1789760780.625397:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.625417:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17536, 17536] 00000008:00010000:2.0:1789760780.625435:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.625440:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.625451:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71827456, 71831552) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.625464:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71827456=0x4480000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.625530:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.625533:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71831552 00000020:00010000:2.0:1789760780.625540:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.625544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.625546:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.625549:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.625552:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.625560:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.625566:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.625583:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760780.627745:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 71827456<>71831552 00000100:00100000:3.0:1789760780.627852:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565474688:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760780.638845:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984844 gen 3 00000008:00010000:2.0:1789760780.638896:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760780.638904:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760780.638915:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760780.638935:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760780.638967:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565474688:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.639081:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17536, 17536] 00000080:00200000:1.0:1789760780.639097:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.639103:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.639120:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71831552 i_blocks 140320 00000020:00200000:1.0:1789760780.639124:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.639136:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.639174:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71831552, count: 4096 00000080:00200000:1.0:1789760780.639182:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71831552, bytes: 4096 00000080:00200000:1.0:1789760780.639192:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.639200:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.639204:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71831552, 71835648) 0 0 00000080:00200000:1.0:1789760780.639208:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.639224:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17537, 17537] 00020000:00200000:1.0:1789760780.639233:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71831552 chunk: [71831552, 71835648) 71835648, 4096 00020000:00200000:1.0:1789760780.639238:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.639258:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17537, 17537] 00000008:00010000:1.0:1789760780.639276:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.639284:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.639296:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71831552, 71835648) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.639310:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71831552=0x4481000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.639374:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.639377:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71835648 00000020:00010000:1.0:1789760780.639384:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.639388:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.639391:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.639394:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.639397:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.639404:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.639412:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.639429:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760780.639579:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 71831552<>71835648 00000100:00100000:1.0:1789760780.639634:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760780.639699:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565474816:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.648613:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984845 gen 3 00000008:00010000:0.0:1789760780.648770:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760780.648779:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.648790:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.648855:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760780.648880:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565474816:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.649185:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17537, 17537] 00000080:00200000:1.0:1789760780.649203:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.649209:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.649240:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71835648 i_blocks 140328 00000020:00200000:1.0:1789760780.649244:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.649255:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.649289:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71835648, count: 4096 00000080:00200000:1.0:1789760780.649298:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71835648, bytes: 4096 00000080:00200000:1.0:1789760780.649310:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.649318:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.649321:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71835648, 71839744) 0 0 00000080:00200000:1.0:1789760780.649326:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.649343:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17538, 17538] 00020000:00200000:1.0:1789760780.649354:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71835648 chunk: [71835648, 71839744) 71839744, 4096 00020000:00200000:1.0:1789760780.649359:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.649379:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17538, 17538] 00000008:00010000:1.0:1789760780.649398:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.649404:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.649415:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71835648, 71839744) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.649428:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71835648=0x4482000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.649625:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.649629:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71839744 00000020:00010000:1.0:1789760780.649638:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.649642:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.649672:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.649675:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.649678:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.649687:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.649694:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.649712:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760780.649912:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 71835648<>71839744 00000100:00100000:0.0:1789760780.649983:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565474944:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.658772:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984846 gen 3 00000008:00010000:0.0:1789760780.658826:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760780.658834:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.658845:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.658901:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760780.658925:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565474944:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.659122:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17538, 17538] 00000080:00200000:1.0:1789760780.659139:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.659144:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.659157:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71839744 i_blocks 140336 00000020:00200000:1.0:1789760780.659160:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.659168:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.659198:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71839744, count: 4096 00000080:00200000:1.0:1789760780.659205:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71839744, bytes: 4096 00000080:00200000:1.0:1789760780.659214:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.659223:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.659226:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71839744, 71843840) 0 0 00000080:00200000:1.0:1789760780.659230:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.659249:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17539, 17539] 00020000:00200000:1.0:1789760780.659258:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71839744 chunk: [71839744, 71843840) 71843840, 4096 00020000:00200000:1.0:1789760780.659264:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.659287:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17539, 17539] 00000008:00010000:1.0:1789760780.659308:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.659315:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.659329:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71839744, 71843840) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.659344:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71839744=0x4483000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.659398:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.659401:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71843840 00000020:00010000:1.0:1789760780.659409:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.659411:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.659414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.659416:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.659418:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.659423:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.659430:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.659445:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760780.659572:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 71839744<>71843840 00000100:00100000:0.0:1789760780.659638:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565475072:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.669963:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984847 gen 3 00000008:00010000:0.0:1789760780.670019:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760780.670028:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.670038:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.670119:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760780.670151:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565475072:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.670244:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17539, 17539] 00000080:00200000:1.0:1789760780.670261:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.670267:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.670282:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71843840 i_blocks 140344 00000020:00200000:1.0:1789760780.670286:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.670295:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.670329:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71843840, count: 4096 00000080:00200000:1.0:1789760780.670338:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71843840, bytes: 4096 00000080:00200000:1.0:1789760780.670351:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.670360:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.670363:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71843840, 71847936) 0 0 00000080:00200000:1.0:1789760780.670367:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.670382:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17540, 17540] 00020000:00200000:1.0:1789760780.670390:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71843840 chunk: [71843840, 71847936) 71847936, 4096 00020000:00200000:1.0:1789760780.670395:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.670413:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17540, 17540] 00000008:00010000:1.0:1789760780.671256:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.671267:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.671288:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71843840, 71847936) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.671305:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71843840=0x4484000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.671382:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.671385:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71847936 00000020:00010000:1.0:1789760780.671395:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.671398:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.671402:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.671405:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.671408:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.671416:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.671423:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.671825:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760780.672065:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 71843840<>71847936 00000100:00100000:0.0:1789760780.672143:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565475200:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.682341:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984848 gen 3 00000008:00010000:0.0:1789760780.682390:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760780.682400:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.682411:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.682533:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760780.682566:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565475200:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.682687:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17540, 17540] 00000080:00200000:1.0:1789760780.682703:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.682709:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.682725:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71847936 i_blocks 140352 00000020:00200000:1.0:1789760780.682728:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.682786:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.682823:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71847936, count: 4096 00000080:00200000:1.0:1789760780.682831:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71847936, bytes: 4096 00000080:00200000:1.0:1789760780.682842:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.682852:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.682854:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71847936, 71852032) 0 0 00000080:00200000:1.0:1789760780.682859:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.682878:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17541, 17541] 00020000:00200000:1.0:1789760780.682888:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71847936 chunk: [71847936, 71852032) 71852032, 4096 00020000:00200000:1.0:1789760780.682893:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.682914:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17541, 17541] 00000008:00010000:1.0:1789760780.682933:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.682939:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.682950:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71847936, 71852032) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.682964:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71847936=0x4485000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.683021:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.683024:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71852032 00000020:00010000:1.0:1789760780.683032:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.683036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.683039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.683043:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.683045:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.683053:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.683061:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.683078:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760780.683201:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 71847936<>71852032 00000100:00100000:0.0:1789760780.683298:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565475328:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.696831:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984849 gen 3 00000008:00010000:0.0:1789760780.696889:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760780.696898:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.696910:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.696933:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760780.696962:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565475328:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.697216:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17541, 17541] 00000080:00200000:1.0:1789760780.697235:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.697241:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.697258:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71852032 i_blocks 140360 00000020:00200000:1.0:1789760780.697261:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.697273:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.697308:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71852032, count: 4096 00000080:00200000:1.0:1789760780.697316:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71852032, bytes: 4096 00000080:00200000:1.0:1789760780.697326:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.697335:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.697338:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71852032, 71856128) 0 0 00000080:00200000:1.0:1789760780.697343:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.697360:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17542, 17542] 00020000:00200000:1.0:1789760780.697369:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71852032 chunk: [71852032, 71856128) 71856128, 4096 00020000:00200000:1.0:1789760780.697374:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.697394:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17542, 17542] 00000008:00010000:1.0:1789760780.697413:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.697420:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.697507:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71852032, 71856128) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.697524:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71852032=0x4486000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.697585:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.697588:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71856128 00000020:00010000:1.0:1789760780.697596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.697600:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.697603:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.697606:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.697609:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.697616:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.697623:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.697641:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760780.697809:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 71852032<>71856128 00000100:00100000:0.0:1789760780.697913:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565475456:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.707266:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984850 gen 3 00000008:00010000:0.0:1789760780.707320:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760780.707329:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.707341:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.707360:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760780.707390:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565475456:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.707567:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17542, 17542] 00000080:00200000:1.0:1789760780.707809:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.707816:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.707968:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71856128 i_blocks 140368 00000020:00200000:2.0:1789760780.707974:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.707985:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.708022:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71856128, count: 4096 00000080:00200000:2.0:1789760780.708030:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71856128, bytes: 4096 00000080:00200000:2.0:1789760780.708041:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.708050:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.708053:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71856128, 71860224) 0 0 00000080:00200000:2.0:1789760780.708057:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.708074:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17543, 17543] 00020000:00200000:2.0:1789760780.708083:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71856128 chunk: [71856128, 71860224) 71860224, 4096 00020000:00200000:2.0:1789760780.708088:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.708316:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17543, 17543] 00000008:00010000:2.0:1789760780.708337:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.708342:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.708355:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71856128, 71860224) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.708368:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71856128=0x4487000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.708565:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.708568:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71860224 00000020:00010000:2.0:1789760780.708577:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.708684:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.708688:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.708691:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.708694:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.708703:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.708711:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.708766:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760780.708903:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 71856128<>71860224 00000100:00100000:2.0:1789760780.708956:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760780.708976:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565475584:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760780.717812:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984851 gen 3 00000008:00010000:2.0:1789760780.717866:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760780.717875:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760780.717886:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760780.717915:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760780.717948:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565475584:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760780.718072:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17543, 17543] 00000080:00200000:0.0:1789760780.718091:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.718098:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.718120:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71860224 i_blocks 140376 00000020:00200000:0.0:1789760780.718124:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.718135:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.718173:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71860224, count: 4096 00000080:00200000:0.0:1789760780.718181:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71860224, bytes: 4096 00000080:00200000:0.0:1789760780.718192:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.718201:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.718204:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71860224, 71864320) 0 0 00000080:00200000:0.0:1789760780.718208:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.718233:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17544, 17544] 00020000:00200000:0.0:1789760780.718243:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71860224 chunk: [71860224, 71864320) 71864320, 4096 00020000:00200000:0.0:1789760780.718247:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.718269:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17544, 17544] 00000008:00010000:0.0:1789760780.718287:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.718294:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.718308:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71860224, 71864320) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.718323:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71860224=0x4488000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.718382:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.718385:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71864320 00000020:00010000:0.0:1789760780.718393:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.718397:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.718400:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.718403:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.718407:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.718415:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.718422:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.718806:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.718938:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71860224<>71864320 00000100:00100000:1.0:1789760780.719001:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760780.719079:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565475712:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.727460:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984852 gen 3 00000008:00010000:0.0:1789760780.727523:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.727908:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.727923:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.727935:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.727963:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565475712:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.728184:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17544, 17544] 00000080:00200000:1.0:1789760780.728203:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.728210:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.728226:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71864320 i_blocks 140384 00000020:00200000:1.0:1789760780.728230:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.728242:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.728275:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71864320, count: 4096 00000080:00200000:1.0:1789760780.728283:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71864320, bytes: 4096 00000080:00200000:1.0:1789760780.728293:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.728305:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.728308:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71864320, 71868416) 0 0 00000080:00200000:1.0:1789760780.728312:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.728328:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17545, 17545] 00020000:00200000:1.0:1789760780.728338:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71864320 chunk: [71864320, 71868416) 71868416, 4096 00020000:00200000:1.0:1789760780.728343:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.728365:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17545, 17545] 00000008:00010000:1.0:1789760780.728383:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.728388:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.728399:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71864320, 71868416) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.728412:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71864320=0x4489000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.728466:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.728469:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71868416 00000020:00010000:1.0:1789760780.728476:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.728479:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.728482:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.728485:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.728487:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.728504:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.728512:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.728527:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760780.728987:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 71864320<>71868416 00000100:00100000:0.0:1789760780.729076:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565475840:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.743017:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984853 gen 3 00000008:00010000:0.0:1789760780.743074:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760780.743083:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.743096:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.743126:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760780.743158:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565475840:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.743251:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17545, 17545] 00000080:00200000:2.0:1789760780.743268:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.743275:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.743290:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71868416 i_blocks 140392 00000020:00200000:2.0:1789760780.743294:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.743305:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.743343:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71868416, count: 4096 00000080:00200000:2.0:1789760780.743351:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71868416, bytes: 4096 00000080:00200000:2.0:1789760780.743361:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.743369:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.743371:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71868416, 71872512) 0 0 00000080:00200000:2.0:1789760780.743376:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.743392:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17546, 17546] 00020000:00200000:2.0:1789760780.743401:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71868416 chunk: [71868416, 71872512) 71872512, 4096 00020000:00200000:2.0:1789760780.743405:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.743423:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17546, 17546] 00000008:00010000:2.0:1789760780.743439:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.743445:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.743456:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71868416, 71872512) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.743468:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71868416=0x448a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.743540:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.743543:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71872512 00000020:00010000:2.0:1789760780.743551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.743555:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.743557:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.743561:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.743564:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.743572:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.743579:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.743594:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760780.743751:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 71868416<>71872512 00000100:00100000:3.0:1789760780.743863:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760780.743882:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565475968:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.764177:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984854 gen 3 00000008:00010000:3.0:1789760780.764249:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760780.764258:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.764270:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.764374:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760780.764398:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565475968:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.764538:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17546, 17546] 00000080:00200000:2.0:1789760780.764554:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.764560:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.764575:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71872512 i_blocks 140400 00000020:00200000:2.0:1789760780.764637:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.764665:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.764700:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71872512, count: 4096 00000080:00200000:2.0:1789760780.764707:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71872512, bytes: 4096 00000080:00200000:2.0:1789760780.764718:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.764727:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.764729:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71872512, 71876608) 0 0 00000080:00200000:2.0:1789760780.764734:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.764751:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17547, 17547] 00020000:00200000:2.0:1789760780.764760:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71872512 chunk: [71872512, 71876608) 71876608, 4096 00020000:00200000:2.0:1789760780.764766:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.764786:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17547, 17547] 00000008:00010000:2.0:1789760780.764803:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.764808:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.764818:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71872512, 71876608) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.764831:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71872512=0x448b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.764888:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.764890:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71876608 00000020:00010000:2.0:1789760780.764899:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.764902:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.764905:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.764909:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.764912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.764919:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.764927:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.764942:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760780.765069:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 71872512<>71876608 00000100:00100000:3.0:1789760780.765135:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565476096:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760780.784992:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984855 gen 3 00000008:00010000:2.0:1789760780.785114:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760780.785122:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760780.785134:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760780.785203:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760780.785236:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565476096:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.785393:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17547, 17547] 00000080:00200000:1.0:1789760780.785410:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.785416:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.785502:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71876608 i_blocks 140408 00000020:00200000:1.0:1789760780.785507:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.785517:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.785552:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71876608, count: 4096 00000080:00200000:1.0:1789760780.785560:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71876608, bytes: 4096 00000080:00200000:1.0:1789760780.785571:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.785579:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.785582:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71876608, 71880704) 0 0 00000080:00200000:1.0:1789760780.785586:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.785603:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17548, 17548] 00020000:00200000:1.0:1789760780.785612:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71876608 chunk: [71876608, 71880704) 71880704, 4096 00020000:00200000:1.0:1789760780.785616:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.785638:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17548, 17548] 00000008:00010000:1.0:1789760780.785772:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.785779:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.785792:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71876608, 71880704) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.785807:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71876608=0x448c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.785870:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.785873:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71880704 00000020:00010000:1.0:1789760780.785882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.785885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.785889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.785892:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.785895:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.785903:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.785910:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.785930:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760780.786246:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 71876608<>71880704 00000100:00100000:0.0:1789760780.786334:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565476224:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.801913:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984856 gen 3 00000008:00010000:0.0:1789760780.801965:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760780.801975:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.801988:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.802056:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760780.802101:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565476224:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.805713:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17548, 17548] 00000080:00200000:1.0:1789760780.805744:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.805751:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.805769:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71880704 i_blocks 140416 00000020:00200000:1.0:1789760780.805775:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.805793:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.805827:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71880704, count: 4096 00000080:00200000:1.0:1789760780.805836:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71880704, bytes: 4096 00000080:00200000:1.0:1789760780.805853:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.805862:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.805865:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71880704, 71884800) 0 0 00000080:00200000:1.0:1789760780.805870:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.805887:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17549, 17549] 00020000:00200000:1.0:1789760780.805897:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71880704 chunk: [71880704, 71884800) 71884800, 4096 00020000:00200000:1.0:1789760780.805903:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.805927:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17549, 17549] 00000008:00010000:1.0:1789760780.805947:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.805954:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.805968:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71880704, 71884800) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.805983:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71880704=0x448d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.806056:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.806059:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71884800 00000020:00010000:1.0:1789760780.806067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.806070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.806073:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.806077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.806079:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.806087:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.806094:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.806123:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760780.806259:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 71880704<>71884800 00000100:00100000:0.0:1789760780.806513:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565476352:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.818215:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984857 gen 3 00000008:00010000:0.0:1789760780.818265:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760780.818274:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.818285:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.818308:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000080:00200000:1.0:1789760780.818334:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17549, 17549] 00000080:00200000:1.0:1789760780.818353:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.818360:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.818375:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71884800 i_blocks 140424 00000020:00200000:1.0:1789760780.818379:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.818389:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000100:00100000:0.0:1789760780.818421:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565476352:192.168.203.147@tcp:4:dd.0 00000080:00200008:1.0:1789760780.818577:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71884800, count: 4096 00000080:00200000:1.0:1789760780.818586:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71884800, bytes: 4096 00000080:00200000:1.0:1789760780.818595:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.818604:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.818607:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71884800, 71888896) 0 0 00000080:00200000:1.0:1789760780.818612:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.818629:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17550, 17550] 00020000:00200000:1.0:1789760780.818638:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71884800 chunk: [71884800, 71888896) 71888896, 4096 00020000:00200000:1.0:1789760780.818737:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.818761:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17550, 17550] 00000008:00010000:1.0:1789760780.818780:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.818786:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.818799:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71884800, 71888896) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.818813:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71884800=0x448e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.818942:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.818945:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71888896 00000020:00010000:1.0:1789760780.818955:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.818958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.818961:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.818965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.818967:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.818974:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.818981:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.818998:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760780.819121:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 71884800<>71888896 00000100:00100000:0.0:1789760780.819189:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565476480:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760780.826929:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760780.832561:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984858 gen 3 00000008:00010000:0.0:1789760780.832615:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760780.832624:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.832636:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.832747:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760780.832777:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565476480:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.837695:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17550, 17550] 00000080:00200000:1.0:1789760780.837721:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.837727:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.837745:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71888896 i_blocks 140432 00000020:00200000:1.0:1789760780.837751:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.837768:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.837802:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71888896, count: 4096 00000080:00200000:1.0:1789760780.837810:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71888896, bytes: 4096 00000080:00200000:1.0:1789760780.837826:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.837836:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.837839:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71888896, 71892992) 0 0 00000080:00200000:1.0:1789760780.837843:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.837859:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17551, 17551] 00020000:00200000:1.0:1789760780.837867:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71888896 chunk: [71888896, 71892992) 71892992, 4096 00020000:00200000:1.0:1789760780.837871:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.837891:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17551, 17551] 00000008:00010000:1.0:1789760780.837908:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.837914:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.837925:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71888896, 71892992) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.837940:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71888896=0x448f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.838008:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.838011:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71892992 00000020:00010000:1.0:1789760780.838019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.838023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.838026:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.838030:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.838033:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.838039:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.838045:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.838073:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760780.838198:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 71888896<>71892992 00000100:00100000:0.0:1789760780.838398:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565476608:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.851897:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984859 gen 3 00000008:00010000:0.0:1789760780.851951:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760780.851960:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.851971:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.852156:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760780.852187:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565476608:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.854668:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17551, 17551] 00000080:00200000:1.0:1789760780.854694:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.854700:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760780.854717:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71892992 i_blocks 140440 00000020:00200000:1.0:1789760780.854723:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760780.854789:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760780.854824:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71892992, count: 4096 00000080:00200000:1.0:1789760780.854832:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71892992, bytes: 4096 00000080:00200000:1.0:1789760780.854847:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760780.854857:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760780.854860:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71892992, 71897088) 0 0 00000080:00200000:1.0:1789760780.854864:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760780.854883:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17552, 17552] 00020000:00200000:1.0:1789760780.854893:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71892992 chunk: [71892992, 71897088) 71897088, 4096 00020000:00200000:1.0:1789760780.854897:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760780.854917:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17552, 17552] 00000008:00010000:1.0:1789760780.854933:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760780.854939:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760780.855003:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71892992, 71897088) append 0 append lockpos 0 00000080:00200000:1.0:1789760780.855023:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71892992=0x4490000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760780.855093:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760780.855096:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71897088 00000020:00010000:1.0:1789760780.855104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760780.855107:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.855110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760780.855114:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760780.855117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760780.855125:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760780.855133:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760780.855164:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760780.855402:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 71892992<>71897088 00000100:00100000:0.0:1789760780.855769:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565476736:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.869134:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984860 gen 3 00000008:00010000:0.0:1789760780.869193:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760780.869202:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.869213:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.869336:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760780.869366:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565476736:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760780.869461:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17552, 17552] 00000080:00200000:1.0:1789760780.869478:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760780.869484:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.869586:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71897088 i_blocks 140448 00000020:00200000:0.0:1789760780.869591:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.869600:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.869634:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71897088, count: 4096 00000080:00200000:0.0:1789760780.869642:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71897088, bytes: 4096 00000080:00200000:0.0:1789760780.869729:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.869739:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.869741:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71897088, 71901184) 0 0 00000080:00200000:0.0:1789760780.869745:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.869761:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17553, 17553] 00020000:00200000:0.0:1789760780.869770:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71897088 chunk: [71897088, 71901184) 71901184, 4096 00020000:00200000:0.0:1789760780.869775:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.869793:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17553, 17553] 00000008:00010000:0.0:1789760780.869811:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.869816:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.869828:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71897088, 71901184) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.869841:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71897088=0x4491000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.869892:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.869895:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71901184 00000020:00010000:0.0:1789760780.869905:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.869909:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.869912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.869917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.869920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.869930:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.869937:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.869951:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.870084:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71897088<>71901184 00000100:00100000:1.0:1789760780.870154:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565476864:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.883089:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984861 gen 3 00000008:00010000:0.0:1789760780.883143:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.883152:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.883163:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.883250:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.883282:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565476864:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.885197:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17553, 17553] 00000080:00200000:2.0:1789760780.885321:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.885329:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.885348:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71901184 i_blocks 140456 00000020:00200000:2.0:1789760780.885354:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.885374:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.885412:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71901184, count: 4096 00000080:00200000:2.0:1789760780.885422:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71901184, bytes: 4096 00000080:00200000:2.0:1789760780.885436:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.885446:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.885448:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71901184, 71905280) 0 0 00000080:00200000:2.0:1789760780.885453:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.885468:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17554, 17554] 00020000:00200000:2.0:1789760780.885477:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71901184 chunk: [71901184, 71905280) 71905280, 4096 00020000:00200000:2.0:1789760780.885481:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.885512:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17554, 17554] 00000008:00010000:2.0:1789760780.885530:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.885537:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.885548:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71901184, 71905280) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.885562:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71901184=0x4492000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.885630:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.885633:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71905280 00000020:00010000:2.0:1789760780.885641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.885663:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.885667:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.885670:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.885673:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.885682:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.885689:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.885720:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760780.885856:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 71901184<>71905280 00000100:00100000:3.0:1789760780.886056:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565476992:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.900933:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984862 gen 3 00000008:00010000:3.0:1789760780.900985:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760780.900995:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.901006:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.901030:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760780.901061:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565476992:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.901174:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17554, 17554] 00000080:00200000:2.0:1789760780.901192:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.901198:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.901214:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71905280 i_blocks 140464 00000020:00200000:2.0:1789760780.901218:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.901279:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.901315:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71905280, count: 4096 00000080:00200000:2.0:1789760780.901322:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71905280, bytes: 4096 00000080:00200000:2.0:1789760780.901332:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.901340:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.901342:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71905280, 71909376) 0 0 00000080:00200000:2.0:1789760780.901347:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.901363:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17555, 17555] 00020000:00200000:2.0:1789760780.901372:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71905280 chunk: [71905280, 71909376) 71909376, 4096 00020000:00200000:2.0:1789760780.901377:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.901399:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17555, 17555] 00000008:00010000:2.0:1789760780.901417:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.901423:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.901434:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71905280, 71909376) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.901448:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71905280=0x4493000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.901512:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.901516:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71909376 00000020:00010000:2.0:1789760780.901523:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.901527:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.901530:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.901533:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.901536:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.901544:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.901551:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.901567:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760780.903080:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 71905280<>71909376 00000100:00100000:3.0:1789760780.908128:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565477120:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.925338:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984863 gen 3 00000008:00010000:3.0:1789760780.925387:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760780.925395:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.925406:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.925428:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760780.925466:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565477120:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.925597:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17555, 17555] 00000080:00200000:2.0:1789760780.925614:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.925620:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.925636:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71909376 i_blocks 140472 00000020:00200000:2.0:1789760780.925639:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.925681:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.925715:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71909376, count: 4096 00000080:00200000:2.0:1789760780.925722:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71909376, bytes: 4096 00000080:00200000:2.0:1789760780.925733:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.925741:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.925744:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71909376, 71913472) 0 0 00000080:00200000:2.0:1789760780.925748:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.925767:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17556, 17556] 00020000:00200000:2.0:1789760780.925777:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71909376 chunk: [71909376, 71913472) 71913472, 4096 00020000:00200000:2.0:1789760780.925782:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.925801:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17556, 17556] 00000008:00010000:2.0:1789760780.925819:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.925825:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.925839:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71909376, 71913472) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.925854:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71909376=0x4494000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.925916:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.925918:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71913472 00000020:00010000:2.0:1789760780.925927:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.925931:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.925934:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.925937:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.925940:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.925947:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.925954:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.925971:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760780.926086:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 71909376<>71913472 00000100:00100000:3.0:1789760780.926200:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565477248:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.941049:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984864 gen 3 00000008:00010000:3.0:1789760780.941096:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760780.941104:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.941114:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.941138:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760780.941168:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565477248:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.941290:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17556, 17556] 00000080:00200000:2.0:1789760780.941308:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.941313:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.941329:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71913472 i_blocks 140480 00000020:00200000:2.0:1789760780.941333:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.941344:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.941378:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71913472, count: 4096 00000080:00200000:2.0:1789760780.941386:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71913472, bytes: 4096 00000080:00200000:2.0:1789760780.941395:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.941404:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.941406:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71913472, 71917568) 0 0 00000080:00200000:2.0:1789760780.941411:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.941427:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17557, 17557] 00020000:00200000:2.0:1789760780.942019:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71913472 chunk: [71913472, 71917568) 71917568, 4096 00020000:00200000:2.0:1789760780.942027:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.942049:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17557, 17557] 00000008:00010000:2.0:1789760780.942066:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.942072:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.942085:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71913472, 71917568) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.942099:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71913472=0x4495000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.942164:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.942167:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71917568 00000020:00010000:2.0:1789760780.942176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.942179:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.942182:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.942186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.942189:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.942197:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.942204:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.942225:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760780.942346:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 71913472<>71917568 00000100:00100000:3.0:1789760780.942419:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565477376:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760780.953854:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984865 gen 3 00000008:00010000:3.0:1789760780.953905:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760780.953915:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760780.953927:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760780.953957:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760780.953991:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565477376:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760780.954121:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17557, 17557] 00000080:00200000:0.0:1789760780.954137:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.954144:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.954159:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71917568 i_blocks 140488 00000020:00200000:0.0:1789760780.954163:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.954174:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.954211:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71917568, count: 4096 00000080:00200000:0.0:1789760780.954226:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71917568, bytes: 4096 00000080:00200000:0.0:1789760780.954242:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.954250:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.954252:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71917568, 71921664) 0 0 00000080:00200000:0.0:1789760780.954257:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.954273:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17558, 17558] 00020000:00200000:0.0:1789760780.954281:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71917568 chunk: [71917568, 71921664) 71921664, 4096 00020000:00200000:0.0:1789760780.954286:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.954307:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17558, 17558] 00000008:00010000:0.0:1789760780.954326:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.954332:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.954344:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71917568, 71921664) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.954359:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71917568=0x4496000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.954414:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.954417:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71921664 00000020:00010000:0.0:1789760780.954425:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.954429:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.954432:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.954436:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.954439:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.954447:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.954454:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.954470:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.954613:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71917568<>71921664 00000100:00100000:0.0:1789760780.954707:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760780.954725:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565477504:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760780.971088:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984866 gen 3 00000008:00010000:0.0:1789760780.971136:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760780.971145:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760780.971155:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760780.971184:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760780.971215:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565477504:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760780.971512:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17558, 17558] 00000080:00200000:2.0:1789760780.971527:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760780.971534:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760780.971551:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71921664 i_blocks 140496 00000020:00200000:2.0:1789760780.971555:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760780.971568:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760780.971597:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71921664, count: 4096 00000080:00200000:2.0:1789760780.971604:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71921664, bytes: 4096 00000080:00200000:2.0:1789760780.971616:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760780.971626:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760780.971628:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71921664, 71925760) 0 0 00000080:00200000:2.0:1789760780.971633:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760780.971690:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17559, 17559] 00020000:00200000:2.0:1789760780.971699:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71921664 chunk: [71921664, 71925760) 71925760, 4096 00020000:00200000:2.0:1789760780.971704:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760780.971724:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17559, 17559] 00000008:00010000:2.0:1789760780.971741:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760780.971747:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760780.971758:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71921664, 71925760) append 0 append lockpos 0 00000080:00200000:2.0:1789760780.971772:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71921664=0x4497000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760780.971830:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760780.971833:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71925760 00000020:00010000:2.0:1789760780.971852:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760780.971856:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.971860:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760780.971864:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760780.971868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760780.971877:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760780.971884:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760780.971901:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760780.972017:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 71921664<>71925760 00000100:00100000:3.0:1789760780.972088:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565477632:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760780.982898:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984867 gen 3 00000008:00010000:2.0:1789760780.982955:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760780.982964:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760780.982976:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760780.983061:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760780.983095:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565477632:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760780.983184:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17559, 17559] 00000080:00200000:0.0:1789760780.983202:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760780.983209:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760780.983276:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71925760 i_blocks 140504 00000020:00200000:0.0:1789760780.983282:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760780.983293:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760780.983331:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71925760, count: 4096 00000080:00200000:0.0:1789760780.983339:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71925760, bytes: 4096 00000080:00200000:0.0:1789760780.983351:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760780.983361:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760780.983363:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71925760, 71929856) 0 0 00000080:00200000:0.0:1789760780.983368:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760780.983386:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17560, 17560] 00020000:00200000:0.0:1789760780.983396:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71925760 chunk: [71925760, 71929856) 71929856, 4096 00020000:00200000:0.0:1789760780.983401:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760780.983421:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17560, 17560] 00000008:00010000:0.0:1789760780.983584:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760780.983591:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760780.983606:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71925760, 71929856) append 0 append lockpos 0 00000080:00200000:0.0:1789760780.983620:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71925760=0x4498000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760780.983713:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760780.983716:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71929856 00000020:00010000:0.0:1789760780.983725:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760780.983728:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.983731:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760780.983734:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760780.983737:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760780.983744:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760780.983751:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760780.983770:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760780.983958:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71925760<>71929856 00000100:00100000:1.0:1789760780.984039:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565477760:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.001035:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984868 gen 3 00000008:00010000:0.0:1789760781.001084:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760781.001093:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.001104:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.001130:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760781.001163:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565477760:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.001296:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17560, 17560] 00000080:00200000:2.0:1789760781.001314:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.001321:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.001338:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71929856 i_blocks 140512 00000020:00200000:2.0:1789760781.001341:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.001353:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.001389:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71929856, count: 4096 00000080:00200000:2.0:1789760781.001397:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71929856, bytes: 4096 00000080:00200000:2.0:1789760781.001409:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.001419:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.001422:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71929856, 71933952) 0 0 00000080:00200000:2.0:1789760781.001437:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.001456:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17561, 17561] 00020000:00200000:2.0:1789760781.001466:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71929856 chunk: [71929856, 71933952) 71933952, 4096 00020000:00200000:2.0:1789760781.001471:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.001504:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17561, 17561] 00000008:00010000:2.0:1789760781.001523:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.001529:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.001542:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71929856, 71933952) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.001561:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71929856=0x4499000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.001619:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.001622:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71933952 00000020:00010000:2.0:1789760781.001631:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.001635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.001638:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.001641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.001691:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.001701:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.001708:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.001724:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760781.001875:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 71929856<>71933952 00000100:00100000:2.0:1789760781.001937:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760781.001960:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565477888:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760781.013537:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984869 gen 3 00000008:00010000:2.0:1789760781.013599:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760781.013608:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.013620:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.013685:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760781.013717:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565477888:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760781.013853:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17561, 17561] 00000080:00200000:0.0:1789760781.013874:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760781.013951:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760781.013973:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71933952 i_blocks 140520 00000020:00200000:0.0:1789760781.013978:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760781.013990:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760781.014027:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71933952, count: 4096 00000080:00200000:0.0:1789760781.014036:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71933952, bytes: 4096 00000080:00200000:0.0:1789760781.014046:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760781.014055:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760781.014058:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71933952, 71938048) 0 0 00000080:00200000:0.0:1789760781.014062:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760781.014080:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17562, 17562] 00020000:00200000:0.0:1789760781.014088:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71933952 chunk: [71933952, 71938048) 71938048, 4096 00020000:00200000:0.0:1789760781.014093:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760781.014113:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17562, 17562] 00000008:00010000:0.0:1789760781.014130:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760781.014137:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760781.014149:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71933952, 71938048) append 0 append lockpos 0 00000080:00200000:0.0:1789760781.014163:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71933952=0x449a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760781.014612:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760781.014615:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71938048 00000020:00010000:0.0:1789760781.014626:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760781.014629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.014632:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.014637:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760781.014640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760781.014802:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760781.014811:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760781.014829:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760781.015016:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71933952<>71938048 00000100:00100000:1.0:1789760781.015088:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565478016:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.025227:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984870 gen 3 00000008:00010000:0.0:1789760781.025281:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:0.0:1789760781.026099:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.026114:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.026145:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:0.0:1789760781.026181:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565478016:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.026862:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17562, 17562] 00000080:00200000:2.0:1789760781.026954:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.026961:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.026979:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71938048 i_blocks 140528 00000020:00200000:2.0:1789760781.026983:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.026997:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.027032:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71938048, count: 4096 00000080:00200000:2.0:1789760781.027040:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71938048, bytes: 4096 00000080:00200000:2.0:1789760781.027055:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.027064:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.027067:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71938048, 71942144) 0 0 00000080:00200000:2.0:1789760781.027072:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.027091:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17563, 17563] 00020000:00200000:2.0:1789760781.027099:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71938048 chunk: [71938048, 71942144) 71942144, 4096 00020000:00200000:2.0:1789760781.027104:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.027125:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17563, 17563] 00000008:00010000:2.0:1789760781.027145:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.027151:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.027163:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71938048, 71942144) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.027178:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71938048=0x449b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.027253:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.027255:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71942144 00000020:00010000:2.0:1789760781.027264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.027267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.027270:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.027274:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.027276:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.027284:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.027300:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.027320:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760781.027465:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 71938048<>71942144 00000100:00100000:3.0:1789760781.027580:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760781.027603:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565478144:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.037947:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984871 gen 3 00000008:00010000:3.0:1789760781.038003:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760781.038012:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.038023:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.038049:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760781.038081:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565478144:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.038742:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17563, 17563] 00000080:00200000:2.0:1789760781.038761:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.038767:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.038784:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71942144 i_blocks 140536 00000020:00200000:2.0:1789760781.038788:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.038799:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.038831:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71942144, count: 4096 00000080:00200000:2.0:1789760781.038839:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71942144, bytes: 4096 00000080:00200000:2.0:1789760781.038849:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.038858:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.038860:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71942144, 71946240) 0 0 00000080:00200000:2.0:1789760781.038864:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.038880:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17564, 17564] 00020000:00200000:2.0:1789760781.038889:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71942144 chunk: [71942144, 71946240) 71946240, 4096 00020000:00200000:2.0:1789760781.038893:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.038913:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17564, 17564] 00000008:00010000:2.0:1789760781.038930:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.038936:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.038946:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71942144, 71946240) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.038959:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71942144=0x449c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.039022:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.039026:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71946240 00000020:00010000:2.0:1789760781.039034:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.039038:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.039041:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.039045:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.039047:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.039055:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.039062:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.039080:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760781.039208:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 71942144<>71946240 00000100:00100000:3.0:1789760781.039294:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565478272:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.049810:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984872 gen 3 00000008:00010000:3.0:1789760781.049864:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760781.049873:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.049883:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.050750:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000080:00200000:2.0:1789760781.050786:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17564, 17564] 00000100:00100000:3.0:1789760781.050791:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565478272:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.050804:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.050810:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.050826:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71946240 i_blocks 140544 00000020:00200000:2.0:1789760781.050830:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.050839:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.050873:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71946240, count: 4096 00000080:00200000:2.0:1789760781.050880:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71946240, bytes: 4096 00000080:00200000:2.0:1789760781.050892:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.050900:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.050903:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71946240, 71950336) 0 0 00000080:00200000:2.0:1789760781.050907:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.050925:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17565, 17565] 00020000:00200000:2.0:1789760781.050934:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71946240 chunk: [71946240, 71950336) 71950336, 4096 00020000:00200000:2.0:1789760781.050938:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.050959:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17565, 17565] 00000008:00010000:2.0:1789760781.050976:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.050982:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.050995:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71946240, 71950336) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.051009:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71946240=0x449d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.051066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.051069:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71950336 00000020:00010000:2.0:1789760781.051076:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.051080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.051084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.051087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.051089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.051097:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.051104:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.051120:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760781.051243:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 71946240<>71950336 00000100:00100000:3.0:1789760781.051405:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565478400:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.061944:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984873 gen 3 00000008:00010000:3.0:1789760781.061995:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760781.062005:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.062016:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.062066:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760781.062099:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565478400:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.065968:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17565, 17565] 00000080:00200000:2.0:1789760781.066000:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.066006:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.066025:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71950336 i_blocks 140552 00000020:00200000:2.0:1789760781.066031:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.066048:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.066084:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71950336, count: 4096 00000080:00200000:2.0:1789760781.066093:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71950336, bytes: 4096 00000080:00200000:2.0:1789760781.066108:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.066119:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.066121:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71950336, 71954432) 0 0 00000080:00200000:2.0:1789760781.066126:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.066144:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17566, 17566] 00020000:00200000:2.0:1789760781.066153:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71950336 chunk: [71950336, 71954432) 71954432, 4096 00020000:00200000:2.0:1789760781.066158:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.066180:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17566, 17566] 00000008:00010000:2.0:1789760781.066199:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.066205:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.066219:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71950336, 71954432) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.066235:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71950336=0x449e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.066309:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.066313:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71954432 00000020:00010000:2.0:1789760781.066321:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.066325:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.066327:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.066331:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.066334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.066342:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.066349:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.066374:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760781.067338:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 71950336<>71954432 00000100:00100000:3.0:1789760781.068738:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760781.068777:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565478528:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.082062:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984874 gen 3 00000008:00010000:3.0:1789760781.082114:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760781.082122:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.082133:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.082152:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760781.082182:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565478528:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.082286:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17566, 17566] 00000080:00200000:2.0:1789760781.082303:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.082309:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.082325:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71954432 i_blocks 140560 00000020:00200000:2.0:1789760781.082328:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.082338:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.082368:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71954432, count: 4096 00000080:00200000:2.0:1789760781.082375:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71954432, bytes: 4096 00000080:00200000:2.0:1789760781.082386:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.082395:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.082397:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71954432, 71958528) 0 0 00000080:00200000:2.0:1789760781.082402:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.082421:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17567, 17567] 00020000:00200000:2.0:1789760781.082430:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71954432 chunk: [71954432, 71958528) 71958528, 4096 00020000:00200000:2.0:1789760781.082435:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.082455:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17567, 17567] 00000008:00010000:2.0:1789760781.082472:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.082478:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.082500:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71954432, 71958528) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.082514:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71954432=0x449f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.082573:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.082576:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71958528 00000020:00010000:2.0:1789760781.082585:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.082588:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.082591:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.082596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.082598:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.082606:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.082612:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.082628:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760781.082801:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 71954432<>71958528 00000100:00100000:3.0:1789760781.082906:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760781.082926:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565478656:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.094052:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984875 gen 3 00000008:00010000:3.0:1789760781.094105:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760781.094113:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.094126:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.094149:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760781.094180:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565478656:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.094318:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17567, 17567] 00000080:00200000:2.0:1789760781.094336:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.094341:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.094361:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71958528 i_blocks 140568 00000020:00200000:2.0:1789760781.094365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.094375:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.094409:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71958528, count: 4096 00000080:00200000:2.0:1789760781.094416:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71958528, bytes: 4096 00000080:00200000:2.0:1789760781.094436:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.094445:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.094447:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71958528, 71962624) 0 0 00000080:00200000:2.0:1789760781.094452:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.094471:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17568, 17568] 00020000:00200000:2.0:1789760781.094480:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71958528 chunk: [71958528, 71962624) 71962624, 4096 00020000:00200000:2.0:1789760781.094485:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.094511:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17568, 17568] 00000008:00010000:2.0:1789760781.094529:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.094535:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.094547:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71958528, 71962624) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.094561:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71958528=0x44a0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.094623:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.094626:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71962624 00000020:00010000:2.0:1789760781.094635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.094639:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.094641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.094691:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.094694:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.094701:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.094708:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.094725:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.094850:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 71958528<>71962624 00000100:00100000:3.0:1789760781.094956:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760781.094975:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565478784:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.104167:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984876 gen 3 00000008:00010000:3.0:1789760781.104217:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760781.104226:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.104236:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.104256:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760781.104285:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565478784:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.104466:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17568, 17568] 00000080:00200000:2.0:1789760781.104482:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.104497:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.104512:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71962624 i_blocks 140576 00000020:00200000:2.0:1789760781.104515:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.104525:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.104556:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71962624, count: 4096 00000080:00200000:2.0:1789760781.104564:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71962624, bytes: 4096 00000080:00200000:2.0:1789760781.104814:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.104823:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.104825:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71962624, 71966720) 0 0 00000080:00200000:2.0:1789760781.104829:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.104848:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17569, 17569] 00020000:00200000:2.0:1789760781.104859:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71962624 chunk: [71962624, 71966720) 71966720, 4096 00020000:00200000:2.0:1789760781.104863:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.104884:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17569, 17569] 00000008:00010000:2.0:1789760781.104901:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.104906:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.104917:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71962624, 71966720) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.104930:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71962624=0x44a1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.104989:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.104991:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71966720 00000020:00010000:2.0:1789760781.104999:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.105003:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.105006:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.105009:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.105012:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.105020:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.105026:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.105042:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.105162:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 71962624<>71966720 00000100:00100000:3.0:1789760781.105822:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760781.105844:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565478912:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.122306:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984877 gen 3 00000008:00010000:2.0:1789760781.123357:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760781.123366:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.123377:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.123399:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760781.123430:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565478912:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760781.123481:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17569, 17569] 00000080:00200000:0.0:1789760781.123506:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760781.123512:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760781.123527:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71966720 i_blocks 140584 00000020:00200000:0.0:1789760781.123531:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760781.123541:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760781.123589:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71966720, count: 4096 00000080:00200000:0.0:1789760781.123597:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71966720, bytes: 4096 00000080:00200000:0.0:1789760781.123612:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760781.123621:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760781.123623:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71966720, 71970816) 0 0 00000080:00200000:0.0:1789760781.123628:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760781.123678:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17570, 17570] 00020000:00200000:0.0:1789760781.123688:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71966720 chunk: [71966720, 71970816) 71970816, 4096 00020000:00200000:0.0:1789760781.123693:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760781.123714:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17570, 17570] 00000008:00010000:0.0:1789760781.123732:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760781.123738:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760781.123751:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71966720, 71970816) append 0 append lockpos 0 00000080:00200000:0.0:1789760781.123765:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71966720=0x44a2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760781.123820:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760781.123823:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71970816 00000020:00010000:0.0:1789760781.123831:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760781.123835:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.123839:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.123842:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760781.123846:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760781.123854:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760781.123861:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760781.123877:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760781.124010:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 71966720<>71970816 00000100:00100000:0.0:1789760781.124048:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760781.124064:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565479040:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760781.138271:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984878 gen 3 00000008:00010000:1.0:1789760781.138331:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760781.138341:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760781.138353:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760781.138380:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760781.138416:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565479040:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.138612:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17570, 17570] 00000080:00200000:2.0:1789760781.138633:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.138639:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.138696:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71970816 i_blocks 140592 00000020:00200000:2.0:1789760781.138701:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.138715:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.138752:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71970816, count: 4096 00000080:00200000:2.0:1789760781.138761:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71970816, bytes: 4096 00000080:00200000:2.0:1789760781.138772:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.138780:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.138783:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71970816, 71974912) 0 0 00000080:00200000:2.0:1789760781.138787:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.138803:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17571, 17571] 00020000:00200000:2.0:1789760781.138813:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71970816 chunk: [71970816, 71974912) 71974912, 4096 00020000:00200000:2.0:1789760781.138819:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.139228:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17571, 17571] 00000008:00010000:2.0:1789760781.139247:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.139253:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.139265:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71970816, 71974912) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.139279:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71970816=0x44a3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.139339:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.139342:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71974912 00000020:00010000:2.0:1789760781.139350:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.139354:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.139357:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.139360:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.139363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.139371:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.139378:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.139395:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.140157:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 71970816<>71974912 00000100:00100000:3.0:1789760781.140245:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760781.140370:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565479168:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760781.160857:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984879 gen 3 00000008:00010000:2.0:1789760781.160906:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760781.160914:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.160934:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.161019:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760781.161049:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565479168:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760781.161186:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17571, 17571] 00000080:00200000:3.0:1789760781.161203:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760781.161210:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.161298:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71974912 i_blocks 140600 00000020:00200000:2.0:1789760781.161303:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.161312:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.161346:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71974912, count: 4096 00000080:00200000:2.0:1789760781.161353:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71974912, bytes: 4096 00000080:00200000:2.0:1789760781.161364:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.161372:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.161375:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71974912, 71979008) 0 0 00000080:00200000:2.0:1789760781.161379:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.161394:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17572, 17572] 00020000:00200000:2.0:1789760781.161403:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71974912 chunk: [71974912, 71979008) 71979008, 4096 00020000:00200000:2.0:1789760781.161418:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.161439:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17572, 17572] 00000008:00010000:2.0:1789760781.161456:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.161462:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.161472:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71974912, 71979008) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.161485:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71974912=0x44a4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.161886:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.161890:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71979008 00000020:00010000:2.0:1789760781.161899:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.161902:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.161906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.161910:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.161913:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.161922:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.161979:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.162029:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.162148:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 71974912<>71979008 00000100:00100000:3.0:1789760781.162213:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565479296:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760781.177315:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984880 gen 3 00000008:00010000:2.0:1789760781.177370:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760781.177378:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.177960:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.178048:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760781.178079:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565479296:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.178321:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17572, 17572] 00000080:00200000:1.0:1789760781.178337:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.178343:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.178359:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71979008 i_blocks 140608 00000020:00200000:1.0:1789760781.178363:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.178374:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.178412:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71979008, count: 4096 00000080:00200000:1.0:1789760781.178421:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71979008, bytes: 4096 00000080:00200000:1.0:1789760781.178435:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.178444:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.178446:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71979008, 71983104) 0 0 00000080:00200000:1.0:1789760781.178450:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.178805:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17573, 17573] 00020000:00200000:1.0:1789760781.178816:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71979008 chunk: [71979008, 71983104) 71983104, 4096 00020000:00200000:1.0:1789760781.178821:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.178844:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17573, 17573] 00000008:00010000:1.0:1789760781.178861:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.178868:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.178879:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71979008, 71983104) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.178893:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71979008=0x44a5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.179153:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.179157:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71983104 00000020:00010000:1.0:1789760781.179166:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.179170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.179173:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.179177:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.179180:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.179189:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.179196:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.179221:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760781.179351:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 71979008<>71983104 00000100:00100000:0.0:1789760781.179598:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565479424:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.190174:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984881 gen 3 00000008:00010000:0.0:1789760781.190218:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760781.190226:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.190237:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.190255:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760781.190290:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565479424:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.190402:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17573, 17573] 00000080:00200000:1.0:1789760781.190427:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.190433:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.190447:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71983104 i_blocks 140616 00000020:00200000:1.0:1789760781.190451:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.190461:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.190503:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71983104, count: 4096 00000080:00200000:1.0:1789760781.190511:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71983104, bytes: 4096 00000080:00200000:1.0:1789760781.190522:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.190532:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.190534:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71983104, 71987200) 0 0 00000080:00200000:1.0:1789760781.190538:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.190555:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17574, 17574] 00020000:00200000:1.0:1789760781.190563:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71983104 chunk: [71983104, 71987200) 71987200, 4096 00020000:00200000:1.0:1789760781.190869:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.190892:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17574, 17574] 00000008:00010000:1.0:1789760781.190910:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.191062:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.191078:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71983104, 71987200) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.191092:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71983104=0x44a6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.191150:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.191153:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71987200 00000020:00010000:1.0:1789760781.191160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.191163:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.191166:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.191170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.191173:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.191179:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.191185:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.191205:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760781.191327:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 71983104<>71987200 00000100:00100000:0.0:1789760781.191704:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565479552:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.201224:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984882 gen 3 00000008:00010000:0.0:1789760781.201275:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760781.201285:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.201295:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.201317:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760781.201346:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565479552:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.201457:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17574, 17574] 00000080:00200000:1.0:1789760781.201474:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.201480:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.201512:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71987200 i_blocks 140624 00000020:00200000:1.0:1789760781.201516:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.201526:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.201560:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71987200, count: 4096 00000080:00200000:1.0:1789760781.201567:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71987200, bytes: 4096 00000080:00200000:1.0:1789760781.201577:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.201585:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.201588:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71987200, 71991296) 0 0 00000080:00200000:1.0:1789760781.201592:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.201609:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17575, 17575] 00020000:00200000:1.0:1789760781.201618:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71987200 chunk: [71987200, 71991296) 71991296, 4096 00020000:00200000:1.0:1789760781.201622:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.201641:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17575, 17575] 00000008:00010000:1.0:1789760781.201693:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.201700:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.201712:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71987200, 71991296) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.201725:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71987200=0x44a7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.201782:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.201784:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71991296 00000020:00010000:1.0:1789760781.201792:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.201796:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.201799:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.201803:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.201806:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.201813:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.201820:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.201969:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760781.202092:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 71987200<>71991296 00000100:00100000:0.0:1789760781.202267:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760781.202287:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565479680:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.211385:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984883 gen 3 00000008:00010000:0.0:1789760781.211436:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760781.211445:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.211456:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.211616:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760781.211681:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565479680:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.211826:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17575, 17575] 00000080:00200000:1.0:1789760781.213949:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.213961:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.213979:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71991296 i_blocks 140632 00000020:00200000:1.0:1789760781.213986:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.214003:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.214039:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71991296, count: 4096 00000080:00200000:1.0:1789760781.214047:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71991296, bytes: 4096 00000080:00200000:1.0:1789760781.214062:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.214070:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.214073:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71991296, 71995392) 0 0 00000080:00200000:1.0:1789760781.214077:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.214095:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17576, 17576] 00020000:00200000:1.0:1789760781.214105:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71991296 chunk: [71991296, 71995392) 71995392, 4096 00020000:00200000:1.0:1789760781.214110:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.214131:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17576, 17576] 00000008:00010000:1.0:1789760781.214150:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.214157:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.214168:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71991296, 71995392) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.214182:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71991296=0x44a8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.214247:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.214250:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71995392 00000020:00010000:1.0:1789760781.214258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.214262:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.214265:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.214269:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.214272:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.214280:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.214286:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.214315:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760781.214461:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 71991296<>71995392 00000100:00100000:0.0:1789760781.214627:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565479808:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.222317:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984884 gen 3 00000008:00010000:0.0:1789760781.222362:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760781.222370:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.222379:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.222399:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760781.222437:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565479808:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.222730:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17576, 17576] 00000080:00200000:1.0:1789760781.222747:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.222753:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.222769:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71995392 i_blocks 140640 00000020:00200000:1.0:1789760781.222772:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.222782:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.222816:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71995392, count: 4096 00000080:00200000:1.0:1789760781.222823:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71995392, bytes: 4096 00000080:00200000:1.0:1789760781.222897:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.222906:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.222908:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71995392, 71999488) 0 0 00000080:00200000:1.0:1789760781.222913:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.222928:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17577, 17577] 00020000:00200000:1.0:1789760781.222937:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71995392 chunk: [71995392, 71999488) 71999488, 4096 00020000:00200000:1.0:1789760781.222942:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.222962:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17577, 17577] 00000008:00010000:1.0:1789760781.222980:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.222987:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.222998:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71995392, 71999488) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.223011:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71995392=0x44a9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.223067:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.223070:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 71999488 00000020:00010000:1.0:1789760781.223077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.223081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.223084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.223087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.223090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.223098:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.223105:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.223123:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760781.223247:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 71995392<>71999488 00000100:00100000:0.0:1789760781.223321:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565479936:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.231851:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984885 gen 3 00000008:00010000:0.0:1789760781.231905:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760781.231913:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.231924:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.232028:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760781.232071:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565479936:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.232215:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17577, 17577] 00000080:00200000:1.0:1789760781.232232:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.232238:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.232253:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 71999488 i_blocks 140648 00000020:00200000:1.0:1789760781.232256:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.232266:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.232301:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 71999488, count: 4096 00000080:00200000:1.0:1789760781.232309:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 71999488, bytes: 4096 00000080:00200000:1.0:1789760781.232319:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.232328:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.232331:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [71999488, 72003584) 0 0 00000080:00200000:1.0:1789760781.232335:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.232353:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17578, 17578] 00020000:00200000:1.0:1789760781.232362:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 71999488 chunk: [71999488, 72003584) 72003584, 4096 00020000:00200000:1.0:1789760781.232367:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.232386:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17578, 17578] 00000008:00010000:1.0:1789760781.232402:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.232409:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.232421:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [71999488, 72003584) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.232436:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=71999488=0x44aa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.232869:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.232873:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72003584 00000020:00010000:1.0:1789760781.232884:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.232888:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.232892:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.232895:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.232899:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.232909:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.232916:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.232939:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760781.233150:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 71999488<>72003584 00000100:00100000:0.0:1789760781.233510:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565480064:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.242950:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984886 gen 3 00000008:00010000:0.0:1789760781.242997:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760781.243005:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.243015:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.243075:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760781.243104:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565480064:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.243188:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17578, 17578] 00000080:00200000:1.0:1789760781.243205:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.243212:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.243226:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72003584 i_blocks 140656 00000020:00200000:1.0:1789760781.243230:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.243240:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.243274:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72003584, count: 4096 00000080:00200000:1.0:1789760781.243281:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72003584, bytes: 4096 00000080:00200000:1.0:1789760781.243291:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.243299:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.243301:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72003584, 72007680) 0 0 00000080:00200000:1.0:1789760781.243306:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.243323:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17579, 17579] 00020000:00200000:1.0:1789760781.243331:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72003584 chunk: [72003584, 72007680) 72007680, 4096 00020000:00200000:1.0:1789760781.243336:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.243353:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17579, 17579] 00000008:00010000:1.0:1789760781.243369:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.243376:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.243387:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72003584, 72007680) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.243400:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72003584=0x44ab000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.243531:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.243534:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72007680 00000020:00010000:1.0:1789760781.243543:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.243618:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.243622:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.243625:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.243628:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.243635:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.243641:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.243741:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760781.244045:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 72003584<>72007680 00000100:00100000:0.0:1789760781.244112:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565480192:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.253104:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984887 gen 3 00000008:00010000:0.0:1789760781.253160:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760781.253169:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.253181:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.253250:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760781.253283:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565480192:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.253285:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17579, 17579] 00000080:00200000:1.0:1789760781.253302:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.253307:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.253323:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72007680 i_blocks 140664 00000020:00200000:1.0:1789760781.253326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.253336:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.253376:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72007680, count: 4096 00000080:00200000:1.0:1789760781.253383:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72007680, bytes: 4096 00000080:00200000:1.0:1789760781.253391:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.253398:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.253400:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72007680, 72011776) 0 0 00000080:00200000:1.0:1789760781.253404:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.253418:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17580, 17580] 00020000:00200000:1.0:1789760781.253425:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72007680 chunk: [72007680, 72011776) 72011776, 4096 00020000:00200000:1.0:1789760781.253430:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.253448:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17580, 17580] 00000008:00010000:1.0:1789760781.253466:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.253472:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.253526:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72007680, 72011776) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.253540:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72007680=0x44ac000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.253601:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.253604:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72011776 00000020:00010000:1.0:1789760781.253622:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.253626:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.253629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.253632:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.253635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.253674:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.253682:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.253698:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760781.253820:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 72007680<>72011776 00000100:00100000:0.0:1789760781.253997:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760781.254015:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565480320:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.265510:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984888 gen 3 00000008:00010000:0.0:1789760781.265788:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760781.265798:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.265810:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.265822:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760781.265863:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565480320:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.266699:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17580, 17580] 00000080:00200000:1.0:1789760781.266717:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.266724:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.266742:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72011776 i_blocks 140672 00000020:00200000:1.0:1789760781.266746:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.266756:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.266801:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72011776, count: 4096 00000080:00200000:1.0:1789760781.266808:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72011776, bytes: 4096 00000080:00200000:1.0:1789760781.266821:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.266829:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.266842:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72011776, 72015872) 0 0 00000080:00200000:1.0:1789760781.266847:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.266864:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17581, 17581] 00020000:00200000:1.0:1789760781.266873:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72011776 chunk: [72011776, 72015872) 72015872, 4096 00020000:00200000:1.0:1789760781.266878:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.266897:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17581, 17581] 00000008:00010000:1.0:1789760781.266915:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.266922:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.266935:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72011776, 72015872) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.266948:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72011776=0x44ad000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.267007:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.267010:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72015872 00000020:00010000:1.0:1789760781.267018:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.267021:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.267024:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.267028:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.267031:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.267039:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.267046:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.267064:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760781.267191:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 72011776<>72015872 00000100:00100000:0.0:1789760781.267502:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565480448:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.276940:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984889 gen 3 00000008:00010000:0.0:1789760781.276993:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760781.277002:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.277015:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.277037:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760781.277066:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565480448:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.277694:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17581, 17581] 00000080:00200000:1.0:1789760781.277713:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.277720:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.277737:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72015872 i_blocks 140680 00000020:00200000:1.0:1789760781.277741:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.277752:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.277787:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72015872, count: 4096 00000080:00200000:1.0:1789760781.277795:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72015872, bytes: 4096 00000080:00200000:1.0:1789760781.277974:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.277984:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.277986:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72015872, 72019968) 0 0 00000080:00200000:1.0:1789760781.277991:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.278008:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17582, 17582] 00020000:00200000:1.0:1789760781.278016:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72015872 chunk: [72015872, 72019968) 72019968, 4096 00020000:00200000:1.0:1789760781.278021:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.278040:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17582, 17582] 00000008:00010000:1.0:1789760781.278058:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.278065:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.278076:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72015872, 72019968) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.278090:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72015872=0x44ae000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.278154:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.278157:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72019968 00000020:00010000:1.0:1789760781.278165:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.278169:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.278172:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.278175:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.278179:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.278186:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.278193:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.278212:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760781.278333:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 72015872<>72019968 00000100:00100000:0.0:1789760781.278637:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760781.278698:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565480576:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.289154:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984890 gen 3 00000008:00010000:0.0:1789760781.289209:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760781.289218:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.289228:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.289252:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000080:00200000:1.0:1789760781.289284:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17582, 17582] 00000100:00100000:0.0:1789760781.289291:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565480576:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.289303:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.289308:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.289324:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72019968 i_blocks 140688 00000020:00200000:1.0:1789760781.289328:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.289339:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.289376:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72019968, count: 4096 00000080:00200000:1.0:1789760781.289384:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72019968, bytes: 4096 00000080:00200000:1.0:1789760781.289396:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.289405:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.289408:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72019968, 72024064) 0 0 00000080:00200000:1.0:1789760781.289413:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.289430:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17583, 17583] 00020000:00200000:1.0:1789760781.289445:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72019968 chunk: [72019968, 72024064) 72024064, 4096 00020000:00200000:1.0:1789760781.289450:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.289471:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17583, 17583] 00000008:00010000:1.0:1789760781.289499:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.289507:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.289519:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72019968, 72024064) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.289534:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72019968=0x44af000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.289594:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.289597:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72024064 00000020:00010000:1.0:1789760781.289606:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.289610:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.289613:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.289822:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.289826:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.289907:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.289915:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.289934:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760781.290069:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 72019968<>72024064 00000100:00100000:0.0:1789760781.290178:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760781.290198:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565480704:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.304509:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984891 gen 3 00000008:00010000:0.0:1789760781.304562:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760781.304570:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.304583:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.304606:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760781.304637:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565480704:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.304779:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17583, 17583] 00000080:00200000:1.0:1789760781.304796:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.304803:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.304819:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72024064 i_blocks 140696 00000020:00200000:1.0:1789760781.304823:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.304834:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.304869:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72024064, count: 4096 00000080:00200000:1.0:1789760781.304876:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72024064, bytes: 4096 00000080:00200000:1.0:1789760781.304888:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.304898:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.304900:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72024064, 72028160) 0 0 00000080:00200000:1.0:1789760781.304904:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.304921:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17584, 17584] 00020000:00200000:1.0:1789760781.304930:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72024064 chunk: [72024064, 72028160) 72028160, 4096 00020000:00200000:1.0:1789760781.304934:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.304952:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17584, 17584] 00000008:00010000:1.0:1789760781.304969:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.304976:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.304987:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72024064, 72028160) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.304999:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72024064=0x44b0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.305064:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.305067:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72028160 00000020:00010000:1.0:1789760781.305074:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.305078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.305081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.305085:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.305088:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.305094:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.305101:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.305116:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760781.305233:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 72024064<>72028160 00000100:00100000:0.0:1789760781.305340:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565480832:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.318006:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984892 gen 3 00000008:00010000:0.0:1789760781.318057:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760781.318066:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.318078:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.318102:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760781.318133:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565480832:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.318687:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17584, 17584] 00000080:00200000:1.0:1789760781.318706:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.318712:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.318728:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72028160 i_blocks 140704 00000020:00200000:1.0:1789760781.318731:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.318742:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.318772:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72028160, count: 4096 00000080:00200000:1.0:1789760781.318780:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72028160, bytes: 4096 00000080:00200000:1.0:1789760781.318790:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.318798:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.318801:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72028160, 72032256) 0 0 00000080:00200000:1.0:1789760781.318805:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.318821:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17585, 17585] 00020000:00200000:1.0:1789760781.318839:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72028160 chunk: [72028160, 72032256) 72032256, 4096 00020000:00200000:1.0:1789760781.318845:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.318867:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17585, 17585] 00000008:00010000:1.0:1789760781.318885:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.318893:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.318904:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72028160, 72032256) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.318918:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72028160=0x44b1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.318980:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.318983:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72032256 00000020:00010000:1.0:1789760781.318992:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.318995:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.318998:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.319002:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.319005:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.319011:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.319017:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.319035:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760781.319157:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 72028160<>72032256 00000100:00100000:0.0:1789760781.319249:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565480960:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.332036:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984893 gen 3 00000008:00010000:0.0:1789760781.332090:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760781.332099:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.332111:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.332133:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760781.332163:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565480960:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.335840:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17585, 17585] 00000080:00200000:1.0:1789760781.335869:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.335876:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.335895:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72032256 i_blocks 140712 00000020:00200000:1.0:1789760781.335900:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.335918:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.335952:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72032256, count: 4096 00000080:00200000:1.0:1789760781.335961:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72032256, bytes: 4096 00000080:00200000:1.0:1789760781.335978:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.335987:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.335990:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72032256, 72036352) 0 0 00000080:00200000:1.0:1789760781.335994:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.336012:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17586, 17586] 00020000:00200000:1.0:1789760781.336021:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72032256 chunk: [72032256, 72036352) 72036352, 4096 00020000:00200000:1.0:1789760781.336026:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.336048:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17586, 17586] 00000008:00010000:1.0:1789760781.336127:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.336135:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.336147:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72032256, 72036352) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.336163:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72032256=0x44b2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.336237:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.336241:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72036352 00000020:00010000:1.0:1789760781.336249:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.336253:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.336256:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.336260:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.336263:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.336272:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.336279:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.336303:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760781.336433:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 72032256<>72036352 00000100:00100000:0.0:1789760781.336706:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565481088:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.343965:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984894 gen 3 00000008:00010000:0.0:1789760781.344013:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760781.344021:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.344032:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.344050:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000080:00200000:1.0:1789760781.344538:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17586, 17586] 00000080:00200000:1.0:1789760781.344558:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.344565:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.344756:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72036352 i_blocks 140720 00000020:00200000:1.0:1789760781.344762:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.344772:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.344809:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72036352, count: 4096 00000080:00200000:1.0:1789760781.344818:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72036352, bytes: 4096 00000080:00200000:1.0:1789760781.344829:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.344838:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.344841:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72036352, 72040448) 0 0 00000080:00200000:1.0:1789760781.344845:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.344865:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17587, 17587] 00020000:00200000:1.0:1789760781.344873:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72036352 chunk: [72036352, 72040448) 72040448, 4096 00020000:00200000:1.0:1789760781.344878:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.344897:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17587, 17587] 00000008:00010000:1.0:1789760781.344915:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.344922:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.344934:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72036352, 72040448) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.344948:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72036352=0x44b3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.345009:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.345012:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72040448 00000020:00010000:1.0:1789760781.345019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.345023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.345027:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.345031:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.345034:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.345043:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.345051:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:0.0:1789760781.345344:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565481088:192.168.203.147@tcp:4:dd.0 00000008:00010000:1.0:1789760781.345420:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760781.345576:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 72036352<>72040448 00000100:00100000:0.0:1789760781.349259:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760781.349306:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565481216:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.362300:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984895 gen 3 00000008:00010000:0.0:1789760781.362612:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760781.362621:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.362632:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.362753:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760781.362784:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565481216:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.362961:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17587, 17587] 00000080:00200000:2.0:1789760781.362979:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.362985:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.362999:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72040448 i_blocks 140728 00000020:00200000:2.0:1789760781.363003:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.363013:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.363048:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72040448, count: 4096 00000080:00200000:2.0:1789760781.363055:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72040448, bytes: 4096 00000080:00200000:2.0:1789760781.363067:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.363075:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.363078:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72040448, 72044544) 0 0 00000080:00200000:2.0:1789760781.363082:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.363098:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17588, 17588] 00020000:00200000:2.0:1789760781.363107:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72040448 chunk: [72040448, 72044544) 72044544, 4096 00020000:00200000:2.0:1789760781.363111:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.363131:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17588, 17588] 00000008:00010000:2.0:1789760781.363158:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.363165:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.363529:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72040448, 72044544) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.363544:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72040448=0x44b4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.363605:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.363608:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72044544 00000020:00010000:2.0:1789760781.363616:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.363620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.363623:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.363628:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.363631:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.363639:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.363673:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.363692:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.364023:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 72040448<>72044544 00000100:00100000:2.0:1789760781.364085:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760781.364107:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565481344:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760781.375483:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984896 gen 3 00000008:00010000:2.0:1789760781.375543:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760781.376683:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.376702:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.376736:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760781.376786:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565481344:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760781.376800:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17588, 17588] 00000080:00200000:0.0:1789760781.376831:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760781.376838:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760781.376855:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72044544 i_blocks 140736 00000020:00200000:0.0:1789760781.376859:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760781.376870:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760781.376905:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72044544, count: 4096 00000080:00200000:0.0:1789760781.376913:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72044544, bytes: 4096 00000080:00200000:0.0:1789760781.376927:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760781.376936:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760781.376938:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72044544, 72048640) 0 0 00000080:00200000:0.0:1789760781.376943:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760781.376961:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17589, 17589] 00020000:00200000:0.0:1789760781.376970:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72044544 chunk: [72044544, 72048640) 72048640, 4096 00020000:00200000:0.0:1789760781.376974:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760781.376994:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17589, 17589] 00000008:00010000:0.0:1789760781.377012:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760781.377019:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760781.377030:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72044544, 72048640) append 0 append lockpos 0 00000080:00200000:0.0:1789760781.377044:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72044544=0x44b5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760781.377101:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760781.377104:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72048640 00000020:00010000:0.0:1789760781.377113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760781.377117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.377120:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.377124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760781.377127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760781.377135:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760781.377141:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760781.377157:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760781.377609:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 72044544<>72048640 00000100:00100000:1.0:1789760781.377721:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565481472:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.393704:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984897 gen 3 00000008:00010000:0.0:1789760781.393755:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:0.0:1789760781.393764:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.393776:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.393803:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:0.0:1789760781.393832:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565481472:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.394604:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17589, 17589] 00000080:00200000:2.0:1789760781.394624:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.394631:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.394670:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72048640 i_blocks 140744 00000020:00200000:2.0:1789760781.394675:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.394690:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.394725:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72048640, count: 4096 00000080:00200000:2.0:1789760781.394732:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72048640, bytes: 4096 00000080:00200000:2.0:1789760781.394745:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.394754:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.394757:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72048640, 72052736) 0 0 00000080:00200000:2.0:1789760781.394761:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.394778:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17590, 17590] 00020000:00200000:2.0:1789760781.394788:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72048640 chunk: [72048640, 72052736) 72052736, 4096 00020000:00200000:2.0:1789760781.394793:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.394815:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17590, 17590] 00000008:00010000:2.0:1789760781.394832:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.394838:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.394850:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72048640, 72052736) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.394862:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72048640=0x44b6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.394939:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.394943:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72052736 00000020:00010000:2.0:1789760781.394951:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.394954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.394957:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.394961:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.394964:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.394972:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.394979:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.394998:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.395125:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 72048640<>72052736 00000100:00100000:3.0:1789760781.396484:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760781.396529:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565481600:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.412080:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984898 gen 3 00000008:00010000:3.0:1789760781.412129:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760781.412137:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.412147:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.412168:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000080:00200000:2.0:1789760781.412306:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17590, 17590] 00000080:00200000:2.0:1789760781.412326:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.412332:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.412346:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72052736 i_blocks 140752 00000020:00200000:2.0:1789760781.412350:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.412360:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.412393:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72052736, count: 4096 00000080:00200000:2.0:1789760781.412401:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72052736, bytes: 4096 00000080:00200000:2.0:1789760781.412411:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.412421:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.412434:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72052736, 72056832) 0 0 00000080:00200000:2.0:1789760781.412439:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.412456:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17591, 17591] 00020000:00200000:2.0:1789760781.412465:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72052736 chunk: [72052736, 72056832) 72056832, 4096 00020000:00200000:2.0:1789760781.412469:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.412497:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17591, 17591] 00000008:00010000:2.0:1789760781.412515:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.412521:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.412532:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72052736, 72056832) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.412546:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72052736=0x44b7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.412595:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.412598:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72056832 00000020:00010000:2.0:1789760781.412605:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.412609:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.412612:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.412615:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.412618:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.412625:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.412631:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:3.0:1789760781.412792:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565481600:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760781.412837:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:3.0:1789760781.412954:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 72052736<>72056832 00000100:00100000:3.0:1789760781.413006:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565481728:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.426980:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984899 gen 3 00000008:00010000:3.0:1789760781.427031:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760781.427039:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.427050:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.427078:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760781.427109:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565481728:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.427234:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17591, 17591] 00000080:00200000:2.0:1789760781.427252:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.427257:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.427272:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72056832 i_blocks 140760 00000020:00200000:2.0:1789760781.427276:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.427285:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.427320:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72056832, count: 4096 00000080:00200000:2.0:1789760781.427328:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72056832, bytes: 4096 00000080:00200000:2.0:1789760781.427338:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.427346:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.427349:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72056832, 72060928) 0 0 00000080:00200000:2.0:1789760781.427353:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.427371:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17592, 17592] 00020000:00200000:2.0:1789760781.427380:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72056832 chunk: [72056832, 72060928) 72060928, 4096 00020000:00200000:2.0:1789760781.427385:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.427405:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17592, 17592] 00000008:00010000:2.0:1789760781.427567:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.427575:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.427589:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72056832, 72060928) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.427603:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72056832=0x44b8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.427770:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.427774:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72060928 00000020:00010000:2.0:1789760781.427783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.427787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.427790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.427793:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.427796:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.427804:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.427811:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.427830:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760781.428176:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 72056832<>72060928 00000100:00100000:3.0:1789760781.428333:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565481856:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.439604:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984900 gen 3 00000008:00010000:3.0:1789760781.440009:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760781.440017:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.440028:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.440121:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760781.440152:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565481856:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.440235:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17592, 17592] 00000080:00200000:2.0:1789760781.440251:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.440257:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.440273:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72060928 i_blocks 140768 00000020:00200000:2.0:1789760781.440277:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.440287:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.440322:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72060928, count: 4096 00000080:00200000:2.0:1789760781.440330:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72060928, bytes: 4096 00000080:00200000:2.0:1789760781.440341:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.440349:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.440352:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72060928, 72065024) 0 0 00000080:00200000:2.0:1789760781.440356:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.440374:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17593, 17593] 00020000:00200000:2.0:1789760781.440382:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72060928 chunk: [72060928, 72065024) 72065024, 4096 00020000:00200000:2.0:1789760781.440387:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.440406:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17593, 17593] 00000008:00010000:2.0:1789760781.440425:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.440432:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.440444:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72060928, 72065024) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.440458:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72060928=0x44b9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.440592:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.440595:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72065024 00000020:00010000:2.0:1789760781.440604:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.440608:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.440620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.440624:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.440627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.440636:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.440666:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.440683:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760781.440817:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 72060928<>72065024 00000100:00100000:3.0:1789760781.440889:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565481984:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.451976:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984901 gen 3 00000008:00010000:3.0:1789760781.452030:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760781.452039:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.452052:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.452077:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760781.452107:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565481984:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760781.452236:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17593, 17593] 00000080:00200000:0.0:1789760781.452252:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760781.452258:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760781.452274:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72065024 i_blocks 140776 00000020:00200000:0.0:1789760781.452278:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760781.452287:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760781.452321:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72065024, count: 4096 00000080:00200000:0.0:1789760781.452329:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72065024, bytes: 4096 00000080:00200000:0.0:1789760781.452339:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760781.452348:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760781.452350:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72065024, 72069120) 0 0 00000080:00200000:0.0:1789760781.452354:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760781.452370:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17594, 17594] 00020000:00200000:0.0:1789760781.452379:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72065024 chunk: [72065024, 72069120) 72069120, 4096 00020000:00200000:0.0:1789760781.452384:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760781.452402:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17594, 17594] 00000008:00010000:0.0:1789760781.452431:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760781.452438:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760781.452448:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72065024, 72069120) append 0 append lockpos 0 00000080:00200000:0.0:1789760781.452461:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72065024=0x44ba000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760781.452527:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760781.452530:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72069120 00000020:00010000:0.0:1789760781.452538:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760781.452541:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.452544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.452547:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760781.452550:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760781.452558:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760781.452564:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760781.452591:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760781.452765:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 72065024<>72069120 00000100:00100000:0.0:1789760781.452804:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760781.452821:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565482112:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.466096:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984902 gen 3 00000008:00010000:0.0:1789760781.466149:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:0.0:1789760781.466158:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.466168:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.466203:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:0.0:1789760781.466344:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565482112:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.466508:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17594, 17594] 00000080:00200000:2.0:1789760781.466526:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.466532:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.466548:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72069120 i_blocks 140784 00000020:00200000:2.0:1789760781.466552:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.466562:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.466605:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72069120, count: 4096 00000080:00200000:2.0:1789760781.466612:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72069120, bytes: 4096 00000080:00200000:2.0:1789760781.466624:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.466967:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.466970:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72069120, 72073216) 0 0 00000080:00200000:2.0:1789760781.466975:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.466993:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17595, 17595] 00020000:00200000:2.0:1789760781.467003:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72069120 chunk: [72069120, 72073216) 72073216, 4096 00020000:00200000:2.0:1789760781.467008:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.467029:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17595, 17595] 00000008:00010000:2.0:1789760781.467049:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.467056:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.467068:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72069120, 72073216) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.467082:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72069120=0x44bb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.467143:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.467145:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72073216 00000020:00010000:2.0:1789760781.467155:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.467159:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.467162:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.467166:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.467169:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.467177:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.467184:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.467200:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760781.468175:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 72069120<>72073216 00000100:00100000:3.0:1789760781.468437:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565482240:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.478451:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984903 gen 3 00000008:00010000:3.0:1789760781.478532:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760781.478541:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.478551:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.478575:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760781.478605:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565482240:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.478770:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17595, 17595] 00000080:00200000:2.0:1789760781.478788:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.478794:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.478810:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72073216 i_blocks 140792 00000020:00200000:2.0:1789760781.478815:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.478826:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.478863:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72073216, count: 4096 00000080:00200000:2.0:1789760781.478872:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72073216, bytes: 4096 00000080:00200000:2.0:1789760781.478883:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.478892:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.478895:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72073216, 72077312) 0 0 00000080:00200000:2.0:1789760781.478899:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.478917:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17596, 17596] 00020000:00200000:2.0:1789760781.478926:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72073216 chunk: [72073216, 72077312) 72077312, 4096 00020000:00200000:2.0:1789760781.478931:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.478952:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17596, 17596] 00000008:00010000:2.0:1789760781.478971:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.478978:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.478990:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72073216, 72077312) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.479004:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72073216=0x44bc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.479063:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.479067:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72077312 00000020:00010000:2.0:1789760781.479075:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.479079:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.479082:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.479085:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.479088:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.479095:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.479102:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.479119:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.479245:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 72073216<>72077312 00000100:00100000:3.0:1789760781.479367:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565482368:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.497856:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984904 gen 3 00000008:00010000:3.0:1789760781.497904:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760781.497912:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.497922:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.497952:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760781.497985:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565482368:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.498883:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17596, 17596] 00000080:00200000:2.0:1789760781.498906:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.498913:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.498930:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72077312 i_blocks 140800 00000020:00200000:2.0:1789760781.498934:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.498946:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.498981:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72077312, count: 4096 00000080:00200000:2.0:1789760781.498988:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72077312, bytes: 4096 00000080:00200000:2.0:1789760781.499004:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.499014:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.499017:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72077312, 72081408) 0 0 00000080:00200000:2.0:1789760781.499021:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.499038:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17597, 17597] 00020000:00200000:2.0:1789760781.499047:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72077312 chunk: [72077312, 72081408) 72081408, 4096 00020000:00200000:2.0:1789760781.499052:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.499071:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17597, 17597] 00000008:00010000:2.0:1789760781.499091:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.499097:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.499108:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72077312, 72081408) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.499123:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72077312=0x44bd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.499190:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.499193:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72081408 00000020:00010000:2.0:1789760781.499201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.499205:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.499207:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.499211:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.499214:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.499221:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.499228:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.499247:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.499370:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 72077312<>72081408 00000100:00100000:3.0:1789760781.499458:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565482496:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760781.509939:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984905 gen 3 00000008:00010000:2.0:1789760781.509988:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760781.509995:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.510006:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.510086:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760781.510114:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565482496:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760781.510258:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17597, 17597] 00000080:00200000:0.0:1789760781.510275:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760781.510281:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760781.510297:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72081408 i_blocks 140808 00000020:00200000:0.0:1789760781.510300:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760781.510311:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760781.510346:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72081408, count: 4096 00000080:00200000:0.0:1789760781.510354:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72081408, bytes: 4096 00000080:00200000:0.0:1789760781.510364:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760781.510372:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760781.510375:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72081408, 72085504) 0 0 00000080:00200000:0.0:1789760781.510379:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760781.510396:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17598, 17598] 00020000:00200000:0.0:1789760781.510405:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72081408 chunk: [72081408, 72085504) 72085504, 4096 00020000:00200000:0.0:1789760781.510477:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760781.510593:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17598, 17598] 00000008:00010000:0.0:1789760781.510612:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760781.510618:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760781.510630:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72081408, 72085504) append 0 append lockpos 0 00000080:00200000:0.0:1789760781.510674:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72081408=0x44be000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760781.510735:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760781.510738:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72085504 00000020:00010000:0.0:1789760781.510747:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760781.510751:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.510754:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.510757:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760781.510760:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760781.510769:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760781.510775:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760781.510795:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760781.510984:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 72081408<>72085504 00000100:00100000:1.0:1789760781.511101:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760781.511122:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565482624:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.522939:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984906 gen 3 00000008:00010000:0.0:1789760781.522990:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:0.0:1789760781.522999:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.523010:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.523073:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:0.0:1789760781.523106:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565482624:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.523199:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17598, 17598] 00000080:00200000:1.0:1789760781.523216:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.523222:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.523238:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72085504 i_blocks 140816 00000020:00200000:1.0:1789760781.523241:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.523253:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.523289:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72085504, count: 4096 00000080:00200000:1.0:1789760781.523297:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72085504, bytes: 4096 00000080:00200000:1.0:1789760781.523307:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.523317:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.523319:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72085504, 72089600) 0 0 00000080:00200000:1.0:1789760781.523324:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.523340:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17599, 17599] 00020000:00200000:1.0:1789760781.523350:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72085504 chunk: [72085504, 72089600) 72089600, 4096 00020000:00200000:1.0:1789760781.523355:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.523375:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17599, 17599] 00000008:00010000:1.0:1789760781.523393:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.523399:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.523542:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72085504, 72089600) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.523558:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72085504=0x44bf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.523619:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.523623:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72089600 00000020:00010000:1.0:1789760781.523631:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.523635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.523638:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.523641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.523778:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.523786:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.523793:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.523814:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760781.524013:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 72085504<>72089600 00000100:00100000:0.0:1789760781.524080:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565482752:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.537957:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984907 gen 3 00000008:00010000:0.0:1789760781.538010:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760781.538019:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.538032:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.538105:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760781.538138:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565482752:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.538809:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17599, 17599] 00000080:00200000:1.0:1789760781.538826:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.538832:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.538848:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72089600 i_blocks 140824 00000020:00200000:1.0:1789760781.538851:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.538862:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.538959:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72089600, count: 4096 00000080:00200000:1.0:1789760781.538967:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72089600, bytes: 4096 00000080:00200000:1.0:1789760781.538978:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.538987:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.538989:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72089600, 72093696) 0 0 00000080:00200000:1.0:1789760781.538994:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.539011:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17600, 17600] 00020000:00200000:1.0:1789760781.539020:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72089600 chunk: [72089600, 72093696) 72093696, 4096 00020000:00200000:1.0:1789760781.539025:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.539048:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17600, 17600] 00000008:00010000:1.0:1789760781.539066:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.539072:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.539085:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72089600, 72093696) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.539098:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72089600=0x44c0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.539421:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.539425:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72093696 00000020:00010000:1.0:1789760781.539435:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.539439:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.539442:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.539447:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.539450:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.539458:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.539466:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.539622:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760781.540005:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 72089600<>72093696 00000100:00100000:0.0:1789760781.540060:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760781.540145:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565482880:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.558342:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984908 gen 3 00000008:00010000:0.0:1789760781.558395:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760781.558404:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.558414:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.558445:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760781.558477:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565482880:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.558681:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17600, 17600] 00000080:00200000:2.0:1789760781.558700:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.558706:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.558724:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72093696 i_blocks 140832 00000020:00200000:2.0:1789760781.558729:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.558741:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.558789:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72093696, count: 4096 00000080:00200000:2.0:1789760781.558798:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72093696, bytes: 4096 00000080:00200000:2.0:1789760781.558811:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.558822:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.558825:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72093696, 72097792) 0 0 00000080:00200000:2.0:1789760781.558830:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.558849:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17601, 17601] 00020000:00200000:2.0:1789760781.558859:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72093696 chunk: [72093696, 72097792) 72097792, 4096 00020000:00200000:2.0:1789760781.558865:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.558894:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17601, 17601] 00000008:00010000:2.0:1789760781.558921:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.558929:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.558943:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72093696, 72097792) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.558958:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72093696=0x44c1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.559047:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.559051:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72097792 00000020:00010000:2.0:1789760781.559061:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.559065:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.559068:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.559072:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.559075:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.559082:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.559090:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.559105:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.560183:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 72093696<>72097792 00000100:00100000:2.0:1789760781.561764:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760781.561796:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565483008:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760781.579460:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984909 gen 3 00000008:00010000:2.0:1789760781.579589:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760781.579597:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.579609:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.579637:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760781.579718:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565483008:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760781.579757:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17601, 17601] 00000080:00200000:0.0:1789760781.579777:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760781.579783:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760781.579799:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72097792 i_blocks 140840 00000020:00200000:0.0:1789760781.579803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760781.579814:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760781.579849:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72097792, count: 4096 00000080:00200000:0.0:1789760781.579857:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72097792, bytes: 4096 00000080:00200000:0.0:1789760781.579867:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760781.579876:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760781.579878:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72097792, 72101888) 0 0 00000080:00200000:0.0:1789760781.579882:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760781.579899:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17602, 17602] 00020000:00200000:0.0:1789760781.579908:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72097792 chunk: [72097792, 72101888) 72101888, 4096 00020000:00200000:0.0:1789760781.579912:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760781.579930:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17602, 17602] 00000008:00010000:0.0:1789760781.579949:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760781.579956:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760781.579967:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72097792, 72101888) append 0 append lockpos 0 00000080:00200000:0.0:1789760781.579982:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72097792=0x44c2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760781.583035:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760781.583048:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72101888 00000020:00010000:0.0:1789760781.583067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760781.583072:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.583076:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.583080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760781.583083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760781.583098:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760781.583110:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760781.583148:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760781.585010:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 72097792<>72101888 00000100:00100000:0.0:1789760781.585093:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565483136:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.609121:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984910 gen 3 00000008:00010000:0.0:1789760781.609169:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:0.0:1789760781.609176:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.609185:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.609198:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:0.0:1789760781.609221:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565483136:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.613065:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17602, 17602] 00000080:00200000:1.0:1789760781.613094:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.613101:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.613121:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72101888 i_blocks 140848 00000020:00200000:1.0:1789760781.613126:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.613145:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.613238:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72101888, count: 4096 00000080:00200000:1.0:1789760781.613249:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72101888, bytes: 4096 00000080:00200000:1.0:1789760781.613265:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.613275:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.613278:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72101888, 72105984) 0 0 00000080:00200000:1.0:1789760781.613283:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.613301:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17603, 17603] 00020000:00200000:1.0:1789760781.613310:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72101888 chunk: [72101888, 72105984) 72105984, 4096 00020000:00200000:1.0:1789760781.613315:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.613338:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17603, 17603] 00000008:00010000:1.0:1789760781.613425:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.613433:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.613446:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72101888, 72105984) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.613461:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72101888=0x44c3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.613715:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.613718:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72105984 00000020:00010000:1.0:1789760781.613729:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.613732:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.613736:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.613740:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.613742:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.613752:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.613760:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.613784:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760781.613924:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 72101888<>72105984 00000100:00100000:0.0:1789760781.615129:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483264:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.633716:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984911 gen 3 00000008:00010000:0.0:1789760781.633767:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760781.633776:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.633787:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.633809:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760781.633839:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483264:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.634623:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17603, 17603] 00000080:00200000:1.0:1789760781.634687:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.634693:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.634709:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72105984 i_blocks 140856 00000020:00200000:1.0:1789760781.634714:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.634727:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.634761:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72105984, count: 4096 00000080:00200000:1.0:1789760781.634769:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72105984, bytes: 4096 00000080:00200000:1.0:1789760781.634781:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.634790:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.634793:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72105984, 72110080) 0 0 00000080:00200000:1.0:1789760781.634797:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.634814:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17604, 17604] 00020000:00200000:1.0:1789760781.634824:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72105984 chunk: [72105984, 72110080) 72110080, 4096 00020000:00200000:1.0:1789760781.634829:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.634850:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17604, 17604] 00000008:00010000:1.0:1789760781.634869:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.634875:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.634886:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72105984, 72110080) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.634900:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72105984=0x44c4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.634958:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.634960:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72110080 00000020:00010000:1.0:1789760781.634968:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.634971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.634974:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.634978:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.634980:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.634987:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.634994:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.635013:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760781.635129:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 72105984<>72110080 00000100:00100000:0.0:1789760781.636731:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760781.636767:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483392:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.651178:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984912 gen 3 00000008:00010000:0.0:1789760781.651292:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760781.651301:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.651313:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.651334:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760781.651366:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483392:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.651763:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17604, 17604] 00000080:00200000:1.0:1789760781.651781:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.651788:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.651803:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72110080 i_blocks 140864 00000020:00200000:1.0:1789760781.651807:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.651818:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.651852:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72110080, count: 4096 00000080:00200000:1.0:1789760781.651859:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72110080, bytes: 4096 00000080:00200000:1.0:1789760781.651870:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.651884:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.651886:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72110080, 72114176) 0 0 00000080:00200000:1.0:1789760781.651892:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.651910:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17605, 17605] 00020000:00200000:1.0:1789760781.651919:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72110080 chunk: [72110080, 72114176) 72114176, 4096 00020000:00200000:1.0:1789760781.651924:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.651945:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17605, 17605] 00000008:00010000:1.0:1789760781.651961:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.651967:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.651978:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72110080, 72114176) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.651991:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72110080=0x44c5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.652053:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.652056:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72114176 00000020:00010000:1.0:1789760781.652065:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.652068:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.652071:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.652075:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.652077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.652085:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.652091:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.652109:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760781.652242:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 72110080<>72114176 00000100:00100000:0.0:1789760781.654433:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483520:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.674051:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984913 gen 3 00000008:00010000:0.0:1789760781.674103:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760781.674112:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.674124:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.674147:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760781.674176:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483520:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.676695:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17605, 17605] 00000080:00200000:1.0:1789760781.676722:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.676728:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.676745:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72114176 i_blocks 140872 00000020:00200000:1.0:1789760781.676750:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.676762:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.676795:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72114176, count: 4096 00000080:00200000:1.0:1789760781.676803:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72114176, bytes: 4096 00000080:00200000:1.0:1789760781.676816:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.676826:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.676828:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72114176, 72118272) 0 0 00000080:00200000:1.0:1789760781.676846:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.676864:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17606, 17606] 00020000:00200000:1.0:1789760781.676873:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72114176 chunk: [72114176, 72118272) 72118272, 4096 00020000:00200000:1.0:1789760781.676877:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.676898:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17606, 17606] 00000008:00010000:1.0:1789760781.676915:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.676922:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.676934:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72114176, 72118272) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.676948:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72114176=0x44c6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.677011:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.677014:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72118272 00000020:00010000:1.0:1789760781.677023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.677027:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.677030:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.677034:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.677036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.677044:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.677051:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.677067:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760781.677222:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 72114176<>72118272 00000100:00100000:0.0:1789760781.677436:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760781.677461:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483648:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.685173:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984914 gen 3 00000008:00010000:0.0:1789760781.685232:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760781.685240:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.685251:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.685273:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760781.685303:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483648:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.685415:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17606, 17606] 00000080:00200000:1.0:1789760781.685441:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.685449:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.685464:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72118272 i_blocks 140880 00000020:00200000:1.0:1789760781.685468:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.685479:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.685523:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72118272, count: 4096 00000080:00200000:1.0:1789760781.685531:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72118272, bytes: 4096 00000080:00200000:1.0:1789760781.685540:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.685550:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.685552:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72118272, 72122368) 0 0 00000080:00200000:1.0:1789760781.685557:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.685576:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17607, 17607] 00020000:00200000:1.0:1789760781.685584:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72118272 chunk: [72118272, 72122368) 72122368, 4096 00020000:00200000:1.0:1789760781.685590:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.685610:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17607, 17607] 00000008:00010000:1.0:1789760781.685628:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.685634:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.685685:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72118272, 72122368) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.685700:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72118272=0x44c7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.685759:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.685763:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72122368 00000020:00010000:1.0:1789760781.685772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.685776:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.685779:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.685783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.685786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.685793:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.685800:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.685817:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760781.685936:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 72118272<>72122368 00000100:00100000:0.0:1789760781.686151:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483776:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.697030:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984915 gen 3 00000008:00010000:0.0:1789760781.697079:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760781.697087:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.697098:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.697118:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760781.697147:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483776:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.697324:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17607, 17607] 00000080:00200000:1.0:1789760781.697345:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.697352:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.697368:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72122368 i_blocks 140888 00000020:00200000:1.0:1789760781.697371:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.697380:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.697413:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72122368, count: 4096 00000080:00200000:1.0:1789760781.697494:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72122368, bytes: 4096 00000080:00200000:1.0:1789760781.697508:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.697516:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.697519:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72122368, 72126464) 0 0 00000080:00200000:1.0:1789760781.697523:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.697540:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17608, 17608] 00020000:00200000:1.0:1789760781.697548:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72122368 chunk: [72122368, 72126464) 72126464, 4096 00020000:00200000:1.0:1789760781.697553:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.697573:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17608, 17608] 00000008:00010000:1.0:1789760781.697592:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.697597:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.697608:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72122368, 72126464) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.697623:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72122368=0x44c8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.697726:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.697729:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72126464 00000020:00010000:1.0:1789760781.697737:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.697740:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.697743:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.697747:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.697749:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.697756:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.697763:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.697780:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760781.697899:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 72122368<>72126464 00000100:00100000:0.0:1789760781.697990:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760781.698007:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483904:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.712778:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984916 gen 3 00000008:00010000:0.0:1789760781.712829:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760781.712838:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.712849:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.712870:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760781.712902:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565483904:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.713167:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17608, 17608] 00000080:00200000:1.0:1789760781.713187:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.713194:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.713222:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72126464 i_blocks 140896 00000020:00200000:1.0:1789760781.713226:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.713236:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.713273:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72126464, count: 4096 00000080:00200000:1.0:1789760781.713281:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72126464, bytes: 4096 00000080:00200000:1.0:1789760781.713292:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.713301:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.713303:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72126464, 72130560) 0 0 00000080:00200000:1.0:1789760781.713308:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.713326:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17609, 17609] 00020000:00200000:1.0:1789760781.713336:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72126464 chunk: [72126464, 72130560) 72130560, 4096 00020000:00200000:1.0:1789760781.713341:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.713361:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17609, 17609] 00000008:00010000:1.0:1789760781.713380:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.713386:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.713398:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72126464, 72130560) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.713412:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72126464=0x44c9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.713496:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.713500:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72130560 00000020:00010000:1.0:1789760781.713510:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.713514:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.713518:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.713522:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.713525:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.713534:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.713542:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.713560:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760781.713726:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 72126464<>72130560 00000100:00100000:0.0:1789760781.714055:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565484032:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.729953:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984917 gen 3 00000008:00010000:0.0:1789760781.730005:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760781.730013:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.730024:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.730045:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760781.730074:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565484032:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.730174:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17609, 17609] 00000080:00200000:1.0:1789760781.730191:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.730197:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.730211:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72130560 i_blocks 140904 00000020:00200000:1.0:1789760781.730215:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.730225:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.730259:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72130560, count: 4096 00000080:00200000:1.0:1789760781.730266:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72130560, bytes: 4096 00000080:00200000:1.0:1789760781.730275:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.730283:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.730285:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72130560, 72134656) 0 0 00000080:00200000:1.0:1789760781.730289:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.730305:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17610, 17610] 00020000:00200000:1.0:1789760781.730314:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72130560 chunk: [72130560, 72134656) 72134656, 4096 00020000:00200000:1.0:1789760781.730319:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.730337:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17610, 17610] 00000008:00010000:1.0:1789760781.730353:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.730358:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.730368:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72130560, 72134656) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.730380:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72130560=0x44ca000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.730441:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.730444:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72134656 00000020:00010000:1.0:1789760781.730451:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.730455:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.730458:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.730461:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.730464:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.730471:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.730477:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.730503:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760781.730608:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 72130560<>72134656 00000100:00100000:0.0:1789760781.730846:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760781.730865:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565484160:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.740706:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984918 gen 3 00000008:00010000:0.0:1789760781.740758:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760781.740766:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.740778:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.740802:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760781.740831:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565484160:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.741704:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17610, 17610] 00000080:00200000:1.0:1789760781.741723:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.741729:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.741745:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72134656 i_blocks 140912 00000020:00200000:1.0:1789760781.741750:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.741760:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.741793:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72134656, count: 4096 00000080:00200000:1.0:1789760781.741801:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72134656, bytes: 4096 00000080:00200000:1.0:1789760781.741814:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.741823:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.741825:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72134656, 72138752) 0 0 00000080:00200000:1.0:1789760781.742046:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.742066:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17611, 17611] 00020000:00200000:1.0:1789760781.742075:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72134656 chunk: [72134656, 72138752) 72138752, 4096 00020000:00200000:1.0:1789760781.742081:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.742103:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17611, 17611] 00000008:00010000:1.0:1789760781.742121:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.742127:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.742139:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72134656, 72138752) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.742152:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72134656=0x44cb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.742214:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.742217:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72138752 00000020:00010000:1.0:1789760781.742226:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.742230:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.742233:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.742236:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.742239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.742248:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.742255:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.742277:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760781.742742:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 72134656<>72138752 00000100:00100000:0.0:1789760781.742815:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565484288:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.750796:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984919 gen 3 00000008:00010000:0.0:1789760781.750849:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760781.750857:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.750990:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.751083:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760781.751117:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565484288:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.756158:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17611, 17611] 00000080:00200000:2.0:1789760781.756187:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.756194:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.756213:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72138752 i_blocks 140920 00000020:00200000:2.0:1789760781.756219:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.756304:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.756346:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72138752, count: 4096 00000080:00200000:2.0:1789760781.756355:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72138752, bytes: 4096 00000080:00200000:2.0:1789760781.756369:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.756377:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.756379:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72138752, 72142848) 0 0 00000080:00200000:2.0:1789760781.756383:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.756398:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17612, 17612] 00020000:00200000:2.0:1789760781.756406:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72138752 chunk: [72138752, 72142848) 72142848, 4096 00020000:00200000:2.0:1789760781.756410:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.756428:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17612, 17612] 00000008:00010000:2.0:1789760781.756447:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.756528:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.756542:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72138752, 72142848) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.756569:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72138752=0x44cc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.756809:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.756813:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72142848 00000020:00010000:2.0:1789760781.756823:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.756827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.756830:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.756834:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.756837:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.756847:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.756853:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.756893:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.757020:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 72138752<>72142848 00000100:00100000:3.0:1789760781.757205:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760781.757332:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565484416:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.775510:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984920 gen 3 00000008:00010000:3.0:1789760781.775565:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760781.775574:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.775585:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.775751:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760781.775784:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565484416:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.776880:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17612, 17612] 00000080:00200000:2.0:1789760781.776903:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.776909:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.776928:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72142848 i_blocks 140928 00000020:00200000:2.0:1789760781.776933:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.776945:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.776979:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72142848, count: 4096 00000080:00200000:2.0:1789760781.776987:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72142848, bytes: 4096 00000080:00200000:2.0:1789760781.776999:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.777009:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.777011:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72142848, 72146944) 0 0 00000080:00200000:2.0:1789760781.777015:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.777033:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17613, 17613] 00020000:00200000:2.0:1789760781.777043:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72142848 chunk: [72142848, 72146944) 72146944, 4096 00020000:00200000:2.0:1789760781.777048:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.777068:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17613, 17613] 00000008:00010000:2.0:1789760781.777087:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.777093:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.777106:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72142848, 72146944) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.777120:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72142848=0x44cd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.777377:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.777381:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72146944 00000020:00010000:2.0:1789760781.777391:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.777394:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.777398:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.777401:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.777404:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.777413:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.777421:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.777439:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760781.777588:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 72142848<>72146944 00000100:00100000:3.0:1789760781.777675:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565484544:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.791400:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565484800:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760781.791409:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:af634c28-3647-4f73-bb24-40d7878858fb:2360:1876699565484672:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1789760781.798461:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:6a301392-4ca2-4c43-8653-61d4c47997a9:2360:1876699565484928:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760781.803333:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:af634c28-3647-4f73-bb24-40d7878858fb:2360:1876699565484672:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1789760781.803974:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565484800:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1789760781.803995:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984921 gen 3 00000008:00010000:3.0:1789760781.804031:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760781.804039:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.804049:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.804074:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760781.804093:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:6a301392-4ca2-4c43-8653-61d4c47997a9:2360:1876699565484928:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1789760781.804103:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565484544:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760781.804295:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17613, 17613] 00000080:00200000:0.0:1789760781.804312:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760781.804319:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760781.804943:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72146944 i_blocks 140936 00000020:00200000:0.0:1789760781.804950:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760781.804963:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760781.804999:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72146944, count: 4096 00000080:00200000:0.0:1789760781.805007:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72146944, bytes: 4096 00000080:00200000:0.0:1789760781.805019:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760781.805028:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760781.805031:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72146944, 72151040) 0 0 00000080:00200000:0.0:1789760781.805036:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760781.805053:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17614, 17614] 00020000:00200000:0.0:1789760781.805063:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72146944 chunk: [72146944, 72151040) 72151040, 4096 00020000:00200000:0.0:1789760781.805068:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760781.805088:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17614, 17614] 00000008:00010000:0.0:1789760781.805108:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760781.805114:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760781.805126:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72146944, 72151040) append 0 append lockpos 0 00000080:00200000:0.0:1789760781.805266:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72146944=0x44ce000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760781.806020:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760781.806025:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72151040 00000020:00010000:0.0:1789760781.806037:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760781.806041:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.806044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.806048:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760781.806051:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760781.806061:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760781.806068:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760781.806096:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760781.807098:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 72146944<>72151040 00000100:00100000:1.0:1789760781.807181:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565485056:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760781.820785:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984922 gen 3 00000008:00010000:1.0:1789760781.820835:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760781.820843:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760781.820853:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760781.820876:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760781.820903:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565485056:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.820992:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17614, 17614] 00000080:00200000:2.0:1789760781.821008:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.821013:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.821028:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72151040 i_blocks 140944 00000020:00200000:2.0:1789760781.821032:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.821043:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.821076:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72151040, count: 4096 00000080:00200000:2.0:1789760781.821083:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72151040, bytes: 4096 00000080:00200000:2.0:1789760781.821094:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.821103:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.821106:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72151040, 72155136) 0 0 00000080:00200000:2.0:1789760781.821111:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.821127:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17615, 17615] 00020000:00200000:2.0:1789760781.821135:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72151040 chunk: [72151040, 72155136) 72155136, 4096 00020000:00200000:2.0:1789760781.821140:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.821158:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17615, 17615] 00000008:00010000:2.0:1789760781.821175:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.821181:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.821192:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72151040, 72155136) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.821205:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72151040=0x44cf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.821261:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.821264:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72155136 00000020:00010000:2.0:1789760781.821271:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.821275:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.821279:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.821282:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.821285:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.821292:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.821299:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.821314:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760781.821494:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 72151040<>72155136 00000100:00100000:2.0:1789760781.821561:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760781.821580:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565485184:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760781.835061:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984923 gen 3 00000008:00010000:2.0:1789760781.835110:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760781.835118:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.835128:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.835153:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760781.835182:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565485184:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.835284:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17615, 17615] 00000080:00200000:1.0:1789760781.835300:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.835307:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.835324:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72155136 i_blocks 140952 00000020:00200000:1.0:1789760781.835327:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.835337:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.835371:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72155136, count: 4096 00000080:00200000:1.0:1789760781.835378:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72155136, bytes: 4096 00000080:00200000:1.0:1789760781.835388:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.835397:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.835400:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72155136, 72159232) 0 0 00000080:00200000:1.0:1789760781.835405:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.835434:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17616, 17616] 00020000:00200000:1.0:1789760781.835443:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72155136 chunk: [72155136, 72159232) 72159232, 4096 00020000:00200000:1.0:1789760781.835447:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.835467:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17616, 17616] 00000008:00010000:1.0:1789760781.835495:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.835502:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.835512:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72155136, 72159232) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.835526:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72155136=0x44d0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.835596:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.835599:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72159232 00000020:00010000:1.0:1789760781.835607:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.835611:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.835615:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.835619:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.835622:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.835631:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.835638:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.835679:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760781.835822:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 72155136<>72159232 00000100:00100000:1.0:1789760781.835869:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760781.835888:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565485312:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760781.848227:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984924 gen 3 00000008:00010000:1.0:1789760781.848283:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:1.0:1789760781.848292:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760781.848302:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760781.848333:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:1.0:1789760781.848364:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565485312:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.848613:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17616, 17616] 00000080:00200000:2.0:1789760781.848633:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.848639:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.848696:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72159232 i_blocks 140960 00000020:00200000:2.0:1789760781.848701:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.848711:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.848746:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72159232, count: 4096 00000080:00200000:2.0:1789760781.848754:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72159232, bytes: 4096 00000080:00200000:2.0:1789760781.848765:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.848774:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.848777:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72159232, 72163328) 0 0 00000080:00200000:2.0:1789760781.848782:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.848799:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17617, 17617] 00020000:00200000:2.0:1789760781.848807:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72159232 chunk: [72159232, 72163328) 72163328, 4096 00020000:00200000:2.0:1789760781.848813:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.848832:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17617, 17617] 00000008:00010000:2.0:1789760781.848850:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.848856:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.848869:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72159232, 72163328) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.848883:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72159232=0x44d1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.848940:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.848943:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72163328 00000020:00010000:2.0:1789760781.848952:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.848955:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.848958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.848962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.848965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.848973:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.848979:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.848996:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760781.849112:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 72159232<>72163328 00000100:00100000:3.0:1789760781.849255:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565485440:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760781.853296:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760781.857226:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984925 gen 3 00000008:00010000:3.0:1789760781.857275:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760781.857283:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.857292:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.857314:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760781.857345:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565485440:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.857452:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17617, 17617] 00000080:00200000:2.0:1789760781.857469:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.857476:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.857501:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72163328 i_blocks 140968 00000020:00200000:2.0:1789760781.857506:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.857516:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.857562:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72163328, count: 4096 00000080:00200000:2.0:1789760781.857570:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72163328, bytes: 4096 00000080:00200000:2.0:1789760781.857584:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.857597:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.857599:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72163328, 72167424) 0 0 00000080:00200000:2.0:1789760781.857604:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.857623:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17618, 17618] 00020000:00200000:2.0:1789760781.857632:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72163328 chunk: [72163328, 72167424) 72167424, 4096 00020000:00200000:2.0:1789760781.857638:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.857681:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17618, 17618] 00000008:00010000:2.0:1789760781.857699:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.857704:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.857715:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72163328, 72167424) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.857728:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72163328=0x44d2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.857788:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.857791:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72167424 00000020:00010000:2.0:1789760781.857802:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.857806:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.857809:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.857813:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.857816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.857827:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.857837:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.857852:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760781.857978:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 72163328<>72167424 00000100:00100000:3.0:1789760781.858131:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760781.858150:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565485568:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.872797:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984926 gen 3 00000008:00010000:3.0:1789760781.872848:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760781.872857:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.872868:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.872891:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760781.872921:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565485568:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.873127:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17618, 17618] 00000080:00200000:2.0:1789760781.873147:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.873153:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.873169:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72167424 i_blocks 140976 00000020:00200000:2.0:1789760781.873172:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.873182:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.873216:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72167424, count: 4096 00000080:00200000:2.0:1789760781.873223:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72167424, bytes: 4096 00000080:00200000:2.0:1789760781.873234:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.873242:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.873245:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72167424, 72171520) 0 0 00000080:00200000:2.0:1789760781.873250:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.873266:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17619, 17619] 00020000:00200000:2.0:1789760781.873274:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72167424 chunk: [72167424, 72171520) 72171520, 4096 00020000:00200000:2.0:1789760781.873279:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.873299:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17619, 17619] 00000008:00010000:2.0:1789760781.873318:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.873324:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.873335:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72167424, 72171520) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.873348:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72167424=0x44d3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.873405:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.873524:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72171520 00000020:00010000:2.0:1789760781.873535:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.873539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.873542:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.873545:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.873549:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.873557:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.873564:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.873581:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760781.874008:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 72167424<>72171520 00000100:00100000:3.0:1789760781.874274:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760781.874297:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565485696:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.885576:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984927 gen 3 00000008:00010000:2.0:1789760781.886025:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760781.886034:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.886046:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.886114:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760781.886144:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565485696:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.886345:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17619, 17619] 00000080:00200000:1.0:1789760781.886363:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.886369:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.886386:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72171520 i_blocks 140984 00000020:00200000:1.0:1789760781.886389:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.886399:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.887248:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72171520, count: 4096 00000080:00200000:1.0:1789760781.887262:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72171520, bytes: 4096 00000080:00200000:1.0:1789760781.887276:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.887287:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.887290:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72171520, 72175616) 0 0 00000080:00200000:1.0:1789760781.887295:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.887311:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17620, 17620] 00020000:00200000:1.0:1789760781.887321:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72171520 chunk: [72171520, 72175616) 72175616, 4096 00020000:00200000:1.0:1789760781.887325:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.887346:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17620, 17620] 00000008:00010000:1.0:1789760781.887366:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.887372:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.887386:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72171520, 72175616) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.887400:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72171520=0x44d4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.887469:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.887472:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72175616 00000020:00010000:1.0:1789760781.887480:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.887788:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.887791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.887795:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.887798:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.887809:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.887816:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.887841:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760781.887966:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 72171520<>72175616 00000100:00100000:0.0:1789760781.888190:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565485824:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.900640:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984928 gen 3 00000008:00010000:0.0:1789760781.900718:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760781.900726:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.900738:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.900750:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760781.900778:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565485824:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760781.900926:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17620, 17620] 00000080:00200000:1.0:1789760781.900943:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760781.900949:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760781.900965:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72175616 i_blocks 140992 00000020:00200000:1.0:1789760781.900969:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760781.900980:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760781.901016:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72175616, count: 4096 00000080:00200000:1.0:1789760781.901024:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72175616, bytes: 4096 00000080:00200000:1.0:1789760781.901036:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760781.901046:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760781.901049:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72175616, 72179712) 0 0 00000080:00200000:1.0:1789760781.901053:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760781.901073:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17621, 17621] 00020000:00200000:1.0:1789760781.901083:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72175616 chunk: [72175616, 72179712) 72179712, 4096 00020000:00200000:1.0:1789760781.901087:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760781.901108:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17621, 17621] 00000008:00010000:1.0:1789760781.901127:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760781.901132:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760781.901145:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72175616, 72179712) append 0 append lockpos 0 00000080:00200000:1.0:1789760781.901161:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72175616=0x44d5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760781.901220:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760781.901223:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72179712 00000020:00010000:1.0:1789760781.901233:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760781.901236:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.901239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760781.901244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760781.901247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760781.901254:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760781.901260:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760781.901276:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760781.901397:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 72175616<>72179712 00000100:00100000:0.0:1789760781.902009:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565485952:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.913916:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984929 gen 3 00000008:00010000:0.0:1789760781.913967:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760781.913975:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.913987:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.914013:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760781.914046:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565485952:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.914148:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17621, 17621] 00000080:00200000:2.0:1789760781.914166:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.914172:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.914187:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72179712 i_blocks 141000 00000020:00200000:2.0:1789760781.914191:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.914202:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.914238:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72179712, count: 4096 00000080:00200000:2.0:1789760781.914246:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72179712, bytes: 4096 00000080:00200000:2.0:1789760781.914257:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.914266:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.914269:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72179712, 72183808) 0 0 00000080:00200000:2.0:1789760781.914275:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.914292:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17622, 17622] 00020000:00200000:2.0:1789760781.914301:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72179712 chunk: [72179712, 72183808) 72183808, 4096 00020000:00200000:2.0:1789760781.914306:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.914326:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17622, 17622] 00000008:00010000:2.0:1789760781.914343:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.914348:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.914361:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72179712, 72183808) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.914377:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72179712=0x44d6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.914436:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.914439:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72183808 00000020:00010000:2.0:1789760781.914449:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.914452:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.914455:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.914459:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.914462:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.914470:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.914477:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.914503:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760781.914630:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 72179712<>72183808 00000100:00100000:2.0:1789760781.914711:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760781.914728:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565486080:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760781.927222:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984930 gen 3 00000008:00010000:2.0:1789760781.927276:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760781.927286:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.927296:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.927322:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760781.927356:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565486080:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760781.927383:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17622, 17622] 00000080:00200000:0.0:1789760781.927402:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760781.927409:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760781.927425:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72183808 i_blocks 141008 00000020:00200000:0.0:1789760781.927429:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760781.927441:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760781.927476:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72183808, count: 4096 00000080:00200000:0.0:1789760781.927496:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72183808, bytes: 4096 00000080:00200000:0.0:1789760781.927509:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760781.927519:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760781.927522:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72183808, 72187904) 0 0 00000080:00200000:0.0:1789760781.927527:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760781.927556:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17623, 17623] 00020000:00200000:0.0:1789760781.927566:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72183808 chunk: [72183808, 72187904) 72187904, 4096 00020000:00200000:0.0:1789760781.927571:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760781.927594:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17623, 17623] 00000008:00010000:0.0:1789760781.927613:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760781.927619:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760781.927632:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72183808, 72187904) append 0 append lockpos 0 00000080:00200000:0.0:1789760781.927669:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72183808=0x44d7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760781.927742:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760781.927745:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72187904 00000020:00010000:0.0:1789760781.927753:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760781.927756:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.927760:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.927764:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760781.927767:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760781.927775:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760781.927782:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760781.927798:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760781.927924:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 72183808<>72187904 00000100:00100000:0.0:1789760781.927971:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760781.927986:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565486208:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.941764:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984931 gen 3 00000008:00010000:0.0:1789760781.941819:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760781.941829:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.941841:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.941871:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760781.941900:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565486208:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.942297:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17623, 17623] 00000080:00200000:2.0:1789760781.942315:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.942321:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.942338:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72187904 i_blocks 141016 00000020:00200000:2.0:1789760781.942343:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.942355:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.942391:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72187904, count: 4096 00000080:00200000:2.0:1789760781.942400:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72187904, bytes: 4096 00000080:00200000:2.0:1789760781.942415:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.942436:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.942438:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72187904, 72192000) 0 0 00000080:00200000:2.0:1789760781.942443:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.942462:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17624, 17624] 00020000:00200000:2.0:1789760781.942471:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72187904 chunk: [72187904, 72192000) 72192000, 4096 00020000:00200000:2.0:1789760781.942476:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.942510:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17624, 17624] 00000008:00010000:2.0:1789760781.942530:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.942536:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.942549:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72187904, 72192000) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.942562:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72187904=0x44d8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.942623:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.942626:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72192000 00000020:00010000:2.0:1789760781.942634:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.942637:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.942640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.942670:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.942673:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.942680:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.942688:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.942705:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760781.942840:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 72187904<>72192000 00000100:00100000:3.0:1789760781.942925:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565486336:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760781.952603:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984932 gen 3 00000008:00010000:2.0:1789760781.952697:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760781.952706:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760781.952717:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760781.952733:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760781.952762:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565486336:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760781.952885:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17624, 17624] 00000080:00200000:0.0:1789760781.952902:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760781.952908:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760781.952924:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72192000 i_blocks 141024 00000020:00200000:0.0:1789760781.952928:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760781.952937:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760781.952973:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72192000, count: 4096 00000080:00200000:0.0:1789760781.952981:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72192000, bytes: 4096 00000080:00200000:0.0:1789760781.952993:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760781.953002:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760781.953004:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72192000, 72196096) 0 0 00000080:00200000:0.0:1789760781.953009:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760781.953026:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17625, 17625] 00020000:00200000:0.0:1789760781.953035:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72192000 chunk: [72192000, 72196096) 72196096, 4096 00020000:00200000:0.0:1789760781.953040:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760781.953061:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17625, 17625] 00000008:00010000:0.0:1789760781.953080:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760781.953085:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760781.953097:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72192000, 72196096) append 0 append lockpos 0 00000080:00200000:0.0:1789760781.953111:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72192000=0x44d9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760781.953167:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760781.953170:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72196096 00000020:00010000:0.0:1789760781.953179:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760781.953182:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.953185:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760781.953188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760781.953191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760781.953199:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760781.953206:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760781.953223:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760781.953345:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 72192000<>72196096 00000100:00100000:1.0:1789760781.953435:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565486464:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760781.959711:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984933 gen 3 00000008:00010000:0.0:1789760781.959761:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760781.959770:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760781.959782:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760781.959898:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760781.959929:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565486464:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.961526:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17625, 17625] 00000080:00200000:2.0:1789760781.961553:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.961560:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.961578:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72196096 i_blocks 141032 00000020:00200000:2.0:1789760781.961585:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.961602:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.961640:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72196096, count: 4096 00000080:00200000:2.0:1789760781.961739:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72196096, bytes: 4096 00000080:00200000:2.0:1789760781.961756:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.961766:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.961768:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72196096, 72200192) 0 0 00000080:00200000:2.0:1789760781.961773:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.961791:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17626, 17626] 00020000:00200000:2.0:1789760781.961801:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72196096 chunk: [72196096, 72200192) 72200192, 4096 00020000:00200000:2.0:1789760781.961806:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.961894:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17626, 17626] 00000008:00010000:2.0:1789760781.961916:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.961923:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.961937:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72196096, 72200192) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.961952:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72196096=0x44da000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.962023:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.962027:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72200192 00000020:00010000:2.0:1789760781.962036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.962039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.962042:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.962046:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.962049:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.962058:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.962064:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.962090:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760781.962219:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 72196096<>72200192 00000100:00100000:3.0:1789760781.962307:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565486592:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.971276:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984934 gen 3 00000008:00010000:3.0:1789760781.971325:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760781.971333:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.971344:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.971370:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760781.971399:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565486592:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.971722:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17626, 17626] 00000080:00200000:2.0:1789760781.971738:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.971744:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.971760:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72200192 i_blocks 141040 00000020:00200000:2.0:1789760781.971764:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.971774:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.971810:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72200192, count: 4096 00000080:00200000:2.0:1789760781.971818:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72200192, bytes: 4096 00000080:00200000:2.0:1789760781.971890:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.971901:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.971904:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72200192, 72204288) 0 0 00000080:00200000:2.0:1789760781.971909:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.971927:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17627, 17627] 00020000:00200000:2.0:1789760781.971936:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72200192 chunk: [72200192, 72204288) 72204288, 4096 00020000:00200000:2.0:1789760781.971940:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.971961:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17627, 17627] 00000008:00010000:2.0:1789760781.971979:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.971985:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.971997:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72200192, 72204288) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.972011:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72200192=0x44db000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.972070:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.972073:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72204288 00000020:00010000:2.0:1789760781.972082:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.972086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.972089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.972092:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.972095:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.972102:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.972110:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.972126:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760781.975176:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 72200192<>72204288 00000100:00100000:3.0:1789760781.975356:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565486720:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.986570:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984935 gen 3 00000008:00010000:3.0:1789760781.986621:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760781.986630:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.986642:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.986791:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000080:00200000:2.0:1789760781.986937:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17627, 17627] 00000080:00200000:2.0:1789760781.986955:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.986961:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.986977:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72204288 i_blocks 141048 00000020:00200000:2.0:1789760781.986981:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.986992:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.987025:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72204288, count: 4096 00000080:00200000:2.0:1789760781.987033:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72204288, bytes: 4096 00000080:00200000:2.0:1789760781.987048:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.987057:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.987059:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72204288, 72208384) 0 0 00000080:00200000:2.0:1789760781.987063:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.987080:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17628, 17628] 00020000:00200000:2.0:1789760781.987089:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72204288 chunk: [72204288, 72208384) 72208384, 4096 00020000:00200000:2.0:1789760781.987093:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.987113:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17628, 17628] 00000008:00010000:2.0:1789760781.987130:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.987136:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.987148:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72204288, 72208384) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.987161:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72204288=0x44dc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.987213:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.987216:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72208384 00000020:00010000:2.0:1789760781.987226:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.987230:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.987233:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.987236:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.987239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.987247:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.987254:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:3.0:1789760781.987319:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565486720:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760781.987385:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:3.0:1789760781.987600:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 72204288<>72208384 00000100:00100000:3.0:1789760781.987668:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760781.987684:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565486848:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760781.996037:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984936 gen 3 00000008:00010000:3.0:1789760781.996088:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760781.996097:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760781.996107:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760781.996134:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760781.996162:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565486848:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760781.996713:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17628, 17628] 00000080:00200000:2.0:1789760781.996732:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760781.996739:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760781.996755:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72208384 i_blocks 141056 00000020:00200000:2.0:1789760781.996758:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760781.996768:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760781.996801:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72208384, count: 4096 00000080:00200000:2.0:1789760781.996809:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72208384, bytes: 4096 00000080:00200000:2.0:1789760781.996821:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760781.996839:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760781.996843:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72208384, 72212480) 0 0 00000080:00200000:2.0:1789760781.996848:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760781.996865:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17629, 17629] 00020000:00200000:2.0:1789760781.996875:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72208384 chunk: [72208384, 72212480) 72212480, 4096 00020000:00200000:2.0:1789760781.996880:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760781.996903:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17629, 17629] 00000008:00010000:2.0:1789760781.996922:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760781.996929:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760781.996942:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72208384, 72212480) append 0 append lockpos 0 00000080:00200000:2.0:1789760781.996956:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72208384=0x44dd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760781.997015:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760781.997018:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72212480 00000020:00010000:2.0:1789760781.997026:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760781.997029:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.997033:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760781.997036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760781.997039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760781.997047:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760781.997054:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760781.997072:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760781.997201:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 72208384<>72212480 00000100:00100000:3.0:1789760781.997750:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565486976:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.007915:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984937 gen 3 00000008:00010000:2.0:1789760782.007973:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760782.007982:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.007995:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.008024:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760782.008056:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565486976:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.008184:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17629, 17629] 00000080:00200000:0.0:1789760782.008204:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.008210:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.008230:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72212480 i_blocks 141064 00000020:00200000:0.0:1789760782.008233:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.008245:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.008285:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72212480, count: 4096 00000080:00200000:0.0:1789760782.008293:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72212480, bytes: 4096 00000080:00200000:0.0:1789760782.008305:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.008315:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.008317:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72212480, 72216576) 0 0 00000080:00200000:0.0:1789760782.008322:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.008341:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17630, 17630] 00020000:00200000:0.0:1789760782.008351:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72212480 chunk: [72212480, 72216576) 72216576, 4096 00020000:00200000:0.0:1789760782.008356:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.008376:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17630, 17630] 00000008:00010000:0.0:1789760782.008395:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.008402:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.008414:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72212480, 72216576) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.008444:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72212480=0x44de000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.008512:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.008515:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72216576 00000020:00010000:0.0:1789760782.008523:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.008527:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.008530:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.008534:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.008537:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.008544:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.008551:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.008566:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760782.008913:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 72212480<>72216576 00000100:00100000:1.0:1789760782.008967:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760782.009288:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565487104:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.025151:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984938 gen 3 00000008:00010000:0.0:1789760782.025399:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:0.0:1789760782.025408:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.025508:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.025572:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:0.0:1789760782.025602:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565487104:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.029876:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17630, 17630] 00000080:00200000:1.0:1789760782.029904:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.029911:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.029929:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72216576 i_blocks 141072 00000020:00200000:1.0:1789760782.029934:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.029953:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.029987:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72216576, count: 4096 00000080:00200000:1.0:1789760782.029996:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72216576, bytes: 4096 00000080:00200000:1.0:1789760782.030011:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.030020:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.030023:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72216576, 72220672) 0 0 00000080:00200000:1.0:1789760782.030027:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.030046:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17631, 17631] 00020000:00200000:1.0:1789760782.030056:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72216576 chunk: [72216576, 72220672) 72220672, 4096 00020000:00200000:1.0:1789760782.030061:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.030089:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17631, 17631] 00000008:00010000:1.0:1789760782.030109:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.030115:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.030126:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72216576, 72220672) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.030140:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72216576=0x44df000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.030213:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.030217:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72220672 00000020:00010000:1.0:1789760782.030226:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.030230:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.030233:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.030237:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.030241:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.030251:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.030259:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.030296:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760782.031911:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 72216576<>72220672 00000100:00100000:0.0:1789760782.032052:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565487232:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.043968:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984939 gen 3 00000008:00010000:0.0:1789760782.044020:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760782.044029:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.044039:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.044117:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760782.044149:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565487232:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.045859:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17631, 17631] 00000080:00200000:2.0:1789760782.045883:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.045889:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.045906:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72220672 i_blocks 141080 00000020:00200000:2.0:1789760782.045910:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.045924:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.045956:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72220672, count: 4096 00000080:00200000:2.0:1789760782.045964:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72220672, bytes: 4096 00000080:00200000:2.0:1789760782.045977:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.045986:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.045989:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72220672, 72224768) 0 0 00000080:00200000:2.0:1789760782.045993:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.046009:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17632, 17632] 00020000:00200000:2.0:1789760782.046017:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72220672 chunk: [72220672, 72224768) 72224768, 4096 00020000:00200000:2.0:1789760782.046022:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.046042:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17632, 17632] 00000008:00010000:2.0:1789760782.046059:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.046065:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.046077:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72220672, 72224768) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.046091:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72220672=0x44e0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.046150:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.046153:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72224768 00000020:00010000:2.0:1789760782.046161:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.046164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.046168:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.046171:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.046174:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.046182:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.046188:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.046285:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760782.046726:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 72220672<>72224768 00000100:00100000:3.0:1789760782.046842:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565487360:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.059185:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984940 gen 3 00000008:00010000:3.0:1789760782.059248:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760782.059256:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.059267:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.059294:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760782.059325:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565487360:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.059838:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17632, 17632] 00000080:00200000:2.0:1789760782.059857:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.059863:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.059879:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72224768 i_blocks 141088 00000020:00200000:2.0:1789760782.059883:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.059894:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.059927:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72224768, count: 4096 00000080:00200000:2.0:1789760782.059935:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72224768, bytes: 4096 00000080:00200000:2.0:1789760782.059945:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.059954:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.059956:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72224768, 72228864) 0 0 00000080:00200000:2.0:1789760782.059960:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.059976:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17633, 17633] 00020000:00200000:2.0:1789760782.059985:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72224768 chunk: [72224768, 72228864) 72228864, 4096 00020000:00200000:2.0:1789760782.059990:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.060009:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17633, 17633] 00000008:00010000:2.0:1789760782.060027:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.060034:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.060046:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72224768, 72228864) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.060060:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72224768=0x44e1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.060123:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.060126:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72228864 00000020:00010000:2.0:1789760782.060135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.060138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.060141:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.060145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.060148:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.060156:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.060164:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.060181:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.060321:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 72224768<>72228864 00000100:00100000:3.0:1789760782.060547:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760782.060568:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565487488:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.068251:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984941 gen 3 00000008:00010000:3.0:1789760782.068363:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760782.068370:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.068381:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.068456:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760782.068496:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565487488:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.068634:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17633, 17633] 00000080:00200000:2.0:1789760782.068751:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.068759:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.068775:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72228864 i_blocks 141096 00000020:00200000:2.0:1789760782.068779:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.068789:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.068820:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72228864, count: 4096 00000080:00200000:2.0:1789760782.068827:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72228864, bytes: 4096 00000080:00200000:2.0:1789760782.068836:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.068844:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.068874:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72228864, 72232960) 0 0 00000080:00200000:2.0:1789760782.068880:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.068897:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17634, 17634] 00020000:00200000:2.0:1789760782.068907:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72228864 chunk: [72228864, 72232960) 72232960, 4096 00020000:00200000:2.0:1789760782.068911:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.068931:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17634, 17634] 00000008:00010000:2.0:1789760782.068949:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.068955:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.068967:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72228864, 72232960) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.068982:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72228864=0x44e2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.069526:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.069532:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72232960 00000020:00010000:2.0:1789760782.069544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.069548:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.069551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.069554:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.069558:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.069567:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.069574:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.069598:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.069755:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 72228864<>72232960 00000100:00100000:3.0:1789760782.070454:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565487616:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.083011:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984942 gen 3 00000008:00010000:3.0:1789760782.083062:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760782.083071:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.083081:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.083107:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760782.083139:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565487616:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.083146:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17634, 17634] 00000080:00200000:2.0:1789760782.083166:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.083172:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.083189:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72232960 i_blocks 141104 00000020:00200000:2.0:1789760782.083193:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.083205:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.083242:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72232960, count: 4096 00000080:00200000:2.0:1789760782.083250:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72232960, bytes: 4096 00000080:00200000:2.0:1789760782.083259:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.083268:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.083270:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72232960, 72237056) 0 0 00000080:00200000:2.0:1789760782.083275:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.083292:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17635, 17635] 00020000:00200000:2.0:1789760782.083302:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72232960 chunk: [72232960, 72237056) 72237056, 4096 00020000:00200000:2.0:1789760782.083307:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.083326:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17635, 17635] 00000008:00010000:2.0:1789760782.083343:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.083349:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.083361:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72232960, 72237056) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.083388:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72232960=0x44e3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.083445:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.083448:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72237056 00000020:00010000:2.0:1789760782.083456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.083459:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.083462:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.083465:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.083468:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.083475:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.083493:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.083509:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760782.083683:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 72232960<>72237056 00000100:00100000:3.0:1789760782.083991:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565487744:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.100255:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984943 gen 3 00000008:00010000:3.0:1789760782.100310:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760782.100319:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.100332:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.100351:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760782.100382:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565487744:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.103759:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17635, 17635] 00000080:00200000:2.0:1789760782.103787:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.103794:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.103812:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72237056 i_blocks 141112 00000020:00200000:2.0:1789760782.103817:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.103834:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.103868:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72237056, count: 4096 00000080:00200000:2.0:1789760782.103877:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72237056, bytes: 4096 00000080:00200000:2.0:1789760782.103893:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.103902:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.103904:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72237056, 72241152) 0 0 00000080:00200000:2.0:1789760782.103909:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.103926:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17636, 17636] 00020000:00200000:2.0:1789760782.103936:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72237056 chunk: [72237056, 72241152) 72241152, 4096 00020000:00200000:2.0:1789760782.103942:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.104042:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17636, 17636] 00000008:00010000:2.0:1789760782.104060:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.104067:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.104080:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72237056, 72241152) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.104095:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72237056=0x44e4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.104171:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.104174:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72241152 00000020:00010000:2.0:1789760782.104183:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.104187:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.104190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.104192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.104195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.104203:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.104210:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.104232:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760782.104368:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 72237056<>72241152 00000100:00100000:3.0:1789760782.104473:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565487872:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.120564:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984944 gen 3 00000008:00010000:3.0:1789760782.120615:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760782.120624:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.120636:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.121064:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760782.121096:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565487872:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.121289:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17636, 17636] 00000080:00200000:0.0:1789760782.121306:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.121313:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.121328:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72241152 i_blocks 141120 00000020:00200000:0.0:1789760782.121332:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.121342:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.121378:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72241152, count: 4096 00000080:00200000:0.0:1789760782.121386:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72241152, bytes: 4096 00000080:00200000:0.0:1789760782.121398:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.121407:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.121409:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72241152, 72245248) 0 0 00000080:00200000:0.0:1789760782.121414:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.121571:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17637, 17637] 00020000:00200000:0.0:1789760782.121581:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72241152 chunk: [72241152, 72245248) 72245248, 4096 00020000:00200000:0.0:1789760782.121586:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.121608:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17637, 17637] 00000008:00010000:0.0:1789760782.121626:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.121633:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.121849:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72241152, 72245248) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.121865:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72241152=0x44e5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.122170:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.122174:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72245248 00000020:00010000:0.0:1789760782.122184:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.122187:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.122191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.122195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.122199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.122208:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.122216:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.122234:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760782.122363:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 72241152<>72245248 00000100:00100000:1.0:1789760782.122613:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565488000:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760782.132915:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984945 gen 3 00000008:00010000:1.0:1789760782.133137:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760782.133147:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760782.133159:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760782.133223:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:1.0:1789760782.133254:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565488000:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.133336:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17637, 17637] 00000080:00200000:0.0:1789760782.133353:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.133359:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.133374:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72245248 i_blocks 141128 00000020:00200000:0.0:1789760782.133378:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.133390:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.133426:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72245248, count: 4096 00000080:00200000:0.0:1789760782.133433:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72245248, bytes: 4096 00000080:00200000:0.0:1789760782.133444:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.133531:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.133535:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72245248, 72249344) 0 0 00000080:00200000:0.0:1789760782.133540:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.133557:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17638, 17638] 00020000:00200000:0.0:1789760782.133566:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72245248 chunk: [72245248, 72249344) 72249344, 4096 00020000:00200000:0.0:1789760782.133571:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.133591:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17638, 17638] 00000008:00010000:0.0:1789760782.133608:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.133615:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.133627:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72245248, 72249344) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.133728:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72245248=0x44e6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.133796:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.133799:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72249344 00000020:00010000:0.0:1789760782.133808:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.133812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.133815:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.133818:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.133821:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.133829:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.133836:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.133852:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d300 00000008:00100000:0.0:1789760782.134376:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 72245248<>72249344 00000100:00100000:1.0:1789760782.134472:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565488128:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760782.143247:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984946 gen 3 00000008:00010000:1.0:1789760782.143295:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760782.143303:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760782.143314:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760782.143441:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760782.143471:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565488128:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.143767:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17638, 17638] 00000080:00200000:0.0:1789760782.143784:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.143790:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.143806:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72249344 i_blocks 141136 00000020:00200000:0.0:1789760782.143810:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.143821:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.143856:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72249344, count: 4096 00000080:00200000:0.0:1789760782.143864:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72249344, bytes: 4096 00000080:00200000:0.0:1789760782.143873:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.143881:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.143884:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72249344, 72253440) 0 0 00000080:00200000:0.0:1789760782.143888:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.143905:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17639, 17639] 00020000:00200000:0.0:1789760782.143914:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72249344 chunk: [72249344, 72253440) 72253440, 4096 00020000:00200000:0.0:1789760782.143918:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.143937:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17639, 17639] 00000008:00010000:0.0:1789760782.143954:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.143961:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.143972:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72249344, 72253440) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.143986:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72249344=0x44e7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.144043:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.144046:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72253440 00000020:00010000:0.0:1789760782.144054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.144058:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.144060:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.144064:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.144067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.144074:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.144081:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.144097:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760782.144217:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 72249344<>72253440 00000100:00100000:1.0:1789760782.144290:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565488256:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760782.153408:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984947 gen 3 00000008:00010000:1.0:1789760782.153459:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760782.153467:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760782.153479:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760782.153558:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:1.0:1789760782.153600:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565488256:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.153955:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17639, 17639] 00000080:00200000:0.0:1789760782.153972:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.153979:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.153994:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72253440 i_blocks 141144 00000020:00200000:0.0:1789760782.153998:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.154010:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.154046:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72253440, count: 4096 00000080:00200000:0.0:1789760782.154054:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72253440, bytes: 4096 00000080:00200000:0.0:1789760782.154065:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.154073:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.154076:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72253440, 72257536) 0 0 00000080:00200000:0.0:1789760782.154081:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.154100:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17640, 17640] 00020000:00200000:0.0:1789760782.154109:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72253440 chunk: [72253440, 72257536) 72257536, 4096 00020000:00200000:0.0:1789760782.154113:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.154131:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17640, 17640] 00000008:00010000:0.0:1789760782.154149:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.154155:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.154166:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72253440, 72257536) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.154181:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72253440=0x44e8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.154240:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.154243:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72257536 00000020:00010000:0.0:1789760782.154251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.154254:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.154257:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.154261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.154264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.154272:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.154280:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.154299:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760782.154807:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 72253440<>72257536 00000100:00100000:1.0:1789760782.154882:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565488384:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760782.174937:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984948 gen 3 00000008:00010000:1.0:1789760782.174988:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760782.174997:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760782.175009:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760782.175035:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760782.175064:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565488384:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.175177:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17640, 17640] 00000080:00200000:0.0:1789760782.175195:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.175203:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.175218:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72257536 i_blocks 141152 00000020:00200000:0.0:1789760782.175222:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.175235:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.175268:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72257536, count: 4096 00000080:00200000:0.0:1789760782.175275:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72257536, bytes: 4096 00000080:00200000:0.0:1789760782.175288:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.175296:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.175298:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72257536, 72261632) 0 0 00000080:00200000:0.0:1789760782.175302:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.175318:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17641, 17641] 00020000:00200000:0.0:1789760782.175327:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72257536 chunk: [72257536, 72261632) 72261632, 4096 00020000:00200000:0.0:1789760782.175332:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.175353:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17641, 17641] 00000008:00010000:0.0:1789760782.175371:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.175377:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.175388:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72257536, 72261632) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.175401:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72257536=0x44e9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.175546:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.175550:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72261632 00000020:00010000:0.0:1789760782.175559:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.175563:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.175567:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.175762:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.175766:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.175776:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.175783:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.175801:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760782.175928:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 72257536<>72261632 00000100:00100000:1.0:1789760782.176760:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565488512:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760782.190795:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984949 gen 3 00000008:00010000:1.0:1789760782.190844:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760782.190852:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760782.190863:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760782.190925:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760782.190957:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565488512:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.193768:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17641, 17641] 00000080:00200000:0.0:1789760782.193801:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.193809:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.193830:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72261632 i_blocks 141160 00000020:00200000:0.0:1789760782.193836:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.193856:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.193894:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72261632, count: 4096 00000080:00200000:0.0:1789760782.193904:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72261632, bytes: 4096 00000080:00200000:0.0:1789760782.193918:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.193928:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.193930:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72261632, 72265728) 0 0 00000080:00200000:0.0:1789760782.193935:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.193953:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17642, 17642] 00020000:00200000:0.0:1789760782.193962:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72261632 chunk: [72261632, 72265728) 72265728, 4096 00020000:00200000:0.0:1789760782.193967:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.193991:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17642, 17642] 00000008:00010000:0.0:1789760782.194010:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.194016:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.194030:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72261632, 72265728) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.194045:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72261632=0x44ea000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.194123:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.194126:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72265728 00000020:00010000:0.0:1789760782.194135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.194139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.194142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.194146:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.194149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.194158:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.194165:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.194196:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760782.194332:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 72261632<>72265728 00000100:00100000:1.0:1789760782.194437:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565488640:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760782.219513:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984950 gen 3 00000008:00010000:1.0:1789760782.219561:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760782.219579:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760782.219736:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760782.219749:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760782.219778:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565488640:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.219883:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17642, 17642] 00000080:00200000:0.0:1789760782.219899:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.219905:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.219921:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72265728 i_blocks 141168 00000020:00200000:0.0:1789760782.219925:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.219936:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.219971:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72265728, count: 4096 00000080:00200000:0.0:1789760782.219979:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72265728, bytes: 4096 00000080:00200000:0.0:1789760782.219989:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.219997:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.219999:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72265728, 72269824) 0 0 00000080:00200000:0.0:1789760782.220003:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.220018:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17643, 17643] 00020000:00200000:0.0:1789760782.220027:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72265728 chunk: [72265728, 72269824) 72269824, 4096 00020000:00200000:0.0:1789760782.220031:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.220050:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17643, 17643] 00000008:00010000:0.0:1789760782.220068:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.220074:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.220087:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72265728, 72269824) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.220099:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72265728=0x44eb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.220155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.220158:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72269824 00000020:00010000:0.0:1789760782.220165:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.220169:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.220172:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.220175:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.220179:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.220187:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.220298:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.220318:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dcc0 00000008:00100000:0.0:1789760782.220449:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 72265728<>72269824 00000100:00100000:1.0:1789760782.220527:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565488768:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760782.232154:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984951 gen 3 00000008:00010000:1.0:1789760782.232203:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760782.232211:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760782.232223:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760782.232248:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760782.232277:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565488768:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.232397:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17643, 17643] 00000080:00200000:2.0:1789760782.232413:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.232419:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.232433:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72269824 i_blocks 141176 00000020:00200000:2.0:1789760782.232437:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.232525:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.232562:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72269824, count: 4096 00000080:00200000:2.0:1789760782.232570:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72269824, bytes: 4096 00000080:00200000:2.0:1789760782.232580:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.232684:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.232687:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72269824, 72273920) 0 0 00000080:00200000:2.0:1789760782.232691:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.232708:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17644, 17644] 00020000:00200000:2.0:1789760782.232717:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72269824 chunk: [72269824, 72273920) 72273920, 4096 00020000:00200000:2.0:1789760782.232721:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.232742:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17644, 17644] 00000008:00010000:2.0:1789760782.232759:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.232765:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.232777:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72269824, 72273920) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.232789:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72269824=0x44ec000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.232901:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.232904:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72273920 00000020:00010000:2.0:1789760782.232912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.232915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.232918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.232922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.232925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.232932:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.232939:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.232956:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760782.233076:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 72269824<>72273920 00000100:00100000:3.0:1789760782.233150:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565488896:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.242185:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984952 gen 3 00000008:00010000:3.0:1789760782.242231:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760782.242239:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.242250:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.242272:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760782.242299:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565488896:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.242416:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17644, 17644] 00000080:00200000:2.0:1789760782.242432:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.242438:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.242564:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72273920 i_blocks 141184 00000020:00200000:2.0:1789760782.242569:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.242579:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.242695:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72273920, count: 4096 00000080:00200000:2.0:1789760782.242703:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72273920, bytes: 4096 00000080:00200000:2.0:1789760782.242714:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.242722:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.242724:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72273920, 72278016) 0 0 00000080:00200000:2.0:1789760782.242729:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.242746:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17645, 17645] 00020000:00200000:2.0:1789760782.242755:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72273920 chunk: [72273920, 72278016) 72278016, 4096 00020000:00200000:2.0:1789760782.242760:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.242779:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17645, 17645] 00000008:00010000:2.0:1789760782.242796:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.242802:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.242812:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72273920, 72278016) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.242826:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72273920=0x44ed000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.242876:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.242879:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72278016 00000020:00010000:2.0:1789760782.242887:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.242891:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.242894:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.242897:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.242900:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.242907:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.242914:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.242928:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760782.243045:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 72273920<>72278016 00000100:00100000:3.0:1789760782.244425:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565489024:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.258534:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984953 gen 3 00000008:00010000:3.0:1789760782.258598:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760782.258607:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.258618:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.258640:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760782.258723:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565489024:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.258949:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17645, 17645] 00000080:00200000:2.0:1789760782.258967:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.258973:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.258989:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72278016 i_blocks 141192 00000020:00200000:2.0:1789760782.258993:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.259004:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.259037:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72278016, count: 4096 00000080:00200000:2.0:1789760782.259045:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72278016, bytes: 4096 00000080:00200000:2.0:1789760782.259060:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.259070:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.259072:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72278016, 72282112) 0 0 00000080:00200000:2.0:1789760782.259076:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.259094:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17646, 17646] 00020000:00200000:2.0:1789760782.259104:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72278016 chunk: [72278016, 72282112) 72282112, 4096 00020000:00200000:2.0:1789760782.259109:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.259130:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17646, 17646] 00000008:00010000:2.0:1789760782.259149:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.259155:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.259168:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72278016, 72282112) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.259183:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72278016=0x44ee000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.259241:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.259244:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72282112 00000020:00010000:2.0:1789760782.259253:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.259256:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.259259:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.259263:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.259266:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.259273:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.259279:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.259294:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.260034:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 72278016<>72282112 00000100:00100000:3.0:1789760782.260134:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565489152:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.279292:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984954 gen 3 00000008:00010000:3.0:1789760782.279343:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760782.279353:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.279365:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.279387:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760782.279418:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565489152:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.279533:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17646, 17646] 00000080:00200000:2.0:1789760782.279618:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.279624:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.279642:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72282112 i_blocks 141200 00000020:00200000:2.0:1789760782.279679:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.279691:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.279725:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72282112, count: 4096 00000080:00200000:2.0:1789760782.279733:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72282112, bytes: 4096 00000080:00200000:2.0:1789760782.279745:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.279755:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.279757:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72282112, 72286208) 0 0 00000080:00200000:2.0:1789760782.279762:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.279780:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17647, 17647] 00020000:00200000:2.0:1789760782.279789:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72282112 chunk: [72282112, 72286208) 72286208, 4096 00020000:00200000:2.0:1789760782.279794:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.279816:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17647, 17647] 00000008:00010000:2.0:1789760782.279835:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.279842:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.279854:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72282112, 72286208) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.279868:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72282112=0x44ef000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.279929:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.279931:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72286208 00000020:00010000:2.0:1789760782.279940:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.279944:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.279947:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.279951:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.279953:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.279961:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.279968:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.279983:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.280114:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 72282112<>72286208 00000100:00100000:3.0:1789760782.280699:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565489280:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.297175:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984955 gen 3 00000008:00010000:3.0:1789760782.297231:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760782.297240:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.297252:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.297275:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760782.297307:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565489280:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.297684:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17647, 17647] 00000080:00200000:2.0:1789760782.297702:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.297708:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.297725:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72286208 i_blocks 141208 00000020:00200000:2.0:1789760782.297728:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.297740:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.297773:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72286208, count: 4096 00000080:00200000:2.0:1789760782.297780:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72286208, bytes: 4096 00000080:00200000:2.0:1789760782.297791:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.297800:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.297803:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72286208, 72290304) 0 0 00000080:00200000:2.0:1789760782.297807:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.297823:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17648, 17648] 00020000:00200000:2.0:1789760782.297832:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72286208 chunk: [72286208, 72290304) 72290304, 4096 00020000:00200000:2.0:1789760782.297837:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.297857:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17648, 17648] 00000008:00010000:2.0:1789760782.297878:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.297885:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.297896:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72286208, 72290304) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.297910:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72286208=0x44f0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.297970:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.297972:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72290304 00000020:00010000:2.0:1789760782.297981:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.297984:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.297988:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.297991:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.297994:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.298002:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.298009:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.298025:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760782.298152:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 72286208<>72290304 00000100:00100000:3.0:1789760782.298640:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760782.298697:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565489408:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.311828:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984956 gen 3 00000008:00010000:3.0:1789760782.311878:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760782.311886:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.311895:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.311918:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760782.311946:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565489408:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.314006:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17648, 17648] 00000080:00200000:2.0:1789760782.314032:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.314039:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.314056:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72290304 i_blocks 141216 00000020:00200000:2.0:1789760782.314061:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.314077:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.314111:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72290304, count: 4096 00000080:00200000:2.0:1789760782.314120:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72290304, bytes: 4096 00000080:00200000:2.0:1789760782.314134:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.314145:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.314147:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72290304, 72294400) 0 0 00000080:00200000:2.0:1789760782.314152:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.314169:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17649, 17649] 00020000:00200000:2.0:1789760782.314179:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72290304 chunk: [72290304, 72294400) 72294400, 4096 00020000:00200000:2.0:1789760782.314184:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.314205:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17649, 17649] 00000008:00010000:2.0:1789760782.314223:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.314231:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.314244:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72290304, 72294400) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.314259:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72290304=0x44f1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.314326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.314328:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72294400 00000020:00010000:2.0:1789760782.314337:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.314341:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.314344:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.314348:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.314351:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.314360:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.314366:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.314387:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760782.314537:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 72290304<>72294400 00000100:00100000:3.0:1789760782.314942:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565489536:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.324025:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984957 gen 3 00000008:00010000:3.0:1789760782.324074:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760782.324082:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.324094:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.324117:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760782.324153:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565489536:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.324669:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17649, 17649] 00000080:00200000:2.0:1789760782.324688:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.324694:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.324709:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72294400 i_blocks 141224 00000020:00200000:2.0:1789760782.324713:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.324725:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.324758:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72294400, count: 4096 00000080:00200000:2.0:1789760782.324767:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72294400, bytes: 4096 00000080:00200000:2.0:1789760782.324779:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.324789:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.324792:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72294400, 72298496) 0 0 00000080:00200000:2.0:1789760782.324796:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.324814:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17650, 17650] 00020000:00200000:2.0:1789760782.324823:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72294400 chunk: [72294400, 72298496) 72298496, 4096 00020000:00200000:2.0:1789760782.324827:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.324847:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17650, 17650] 00000008:00010000:2.0:1789760782.324865:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.324872:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.324884:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72294400, 72298496) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.324898:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72294400=0x44f2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.325030:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.325033:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72298496 00000020:00010000:2.0:1789760782.325042:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.325046:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.325049:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.325053:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.325056:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.325066:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.325073:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.325091:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760782.325214:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 72294400<>72298496 00000100:00100000:3.0:1789760782.325302:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565489664:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.344208:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984958 gen 3 00000008:00010000:3.0:1789760782.344261:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760782.344269:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.344283:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.344379:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760782.344412:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565489664:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.349045:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17650, 17650] 00000080:00200000:0.0:1789760782.349075:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.349083:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.349102:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72298496 i_blocks 141232 00000020:00200000:0.0:1789760782.349108:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.349126:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.349165:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72298496, count: 4096 00000080:00200000:0.0:1789760782.349175:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72298496, bytes: 4096 00000080:00200000:0.0:1789760782.349194:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.349204:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.349206:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72298496, 72302592) 0 0 00000080:00200000:0.0:1789760782.349221:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.349239:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17651, 17651] 00020000:00200000:0.0:1789760782.349249:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72298496 chunk: [72298496, 72302592) 72302592, 4096 00020000:00200000:0.0:1789760782.349254:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.349280:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17651, 17651] 00000008:00010000:0.0:1789760782.349299:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.349306:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.349319:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72298496, 72302592) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.349334:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72298496=0x44f3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.349416:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.349419:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72302592 00000020:00010000:0.0:1789760782.349428:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.349432:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.349435:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.349438:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.349442:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.349452:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.349458:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.349502:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.349982:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72298496<>72302592 00000100:00100000:1.0:1789760782.350041:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760782.350828:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565489792:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.361720:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984959 gen 3 00000008:00010000:0.0:1789760782.361779:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.361788:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.361799:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.361822:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.361852:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565489792:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.361950:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17651, 17651] 00000080:00200000:1.0:1789760782.361968:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.361974:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.361989:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72302592 i_blocks 141240 00000020:00200000:1.0:1789760782.361993:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.362004:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.362050:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72302592, count: 4096 00000080:00200000:1.0:1789760782.362059:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72302592, bytes: 4096 00000080:00200000:1.0:1789760782.362071:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.362081:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.362084:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72302592, 72306688) 0 0 00000080:00200000:1.0:1789760782.362088:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.362103:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17652, 17652] 00020000:00200000:1.0:1789760782.362113:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72302592 chunk: [72302592, 72306688) 72306688, 4096 00020000:00200000:1.0:1789760782.362118:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.362136:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17652, 17652] 00000008:00010000:1.0:1789760782.362153:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.362159:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.362170:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72302592, 72306688) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.362186:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72302592=0x44f4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.362244:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.362247:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72306688 00000020:00010000:1.0:1789760782.362255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.362258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.362262:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.362265:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.362269:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.362276:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.362287:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.362304:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760782.362421:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 72302592<>72306688 00000100:00100000:0.0:1789760782.362522:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760782.362539:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565489920:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.371182:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984960 gen 3 00000008:00010000:0.0:1789760782.371233:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760782.371242:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.371252:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.371274:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760782.371304:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565489920:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.371877:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17652, 17652] 00000080:00200000:1.0:1789760782.371895:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.371900:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.371916:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72306688 i_blocks 141248 00000020:00200000:1.0:1789760782.371919:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.371929:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.371962:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72306688, count: 4096 00000080:00200000:1.0:1789760782.371969:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72306688, bytes: 4096 00000080:00200000:1.0:1789760782.371980:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.371988:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.371991:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72306688, 72310784) 0 0 00000080:00200000:1.0:1789760782.371996:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.372013:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17653, 17653] 00020000:00200000:1.0:1789760782.372022:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72306688 chunk: [72306688, 72310784) 72310784, 4096 00020000:00200000:1.0:1789760782.372027:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.372058:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17653, 17653] 00000008:00010000:1.0:1789760782.372076:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.372082:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.372095:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72306688, 72310784) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.372111:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72306688=0x44f5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.372174:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.372178:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72310784 00000020:00010000:1.0:1789760782.372185:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.372189:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.372192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.372196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.372199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.372206:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.372214:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.372234:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760782.372469:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 72306688<>72310784 00000100:00100000:0.0:1789760782.372597:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760782.372619:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565490048:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.382124:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984961 gen 3 00000008:00010000:0.0:1789760782.382178:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760782.382186:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.382198:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.382221:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760782.382251:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565490048:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.382413:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17653, 17653] 00000080:00200000:1.0:1789760782.382431:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.382437:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.382453:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72310784 i_blocks 141256 00000020:00200000:1.0:1789760782.382457:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.382468:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.382517:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72310784, count: 4096 00000080:00200000:1.0:1789760782.382526:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72310784, bytes: 4096 00000080:00200000:1.0:1789760782.382539:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.382548:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.382562:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72310784, 72314880) 0 0 00000080:00200000:1.0:1789760782.382566:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.382584:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17654, 17654] 00020000:00200000:1.0:1789760782.382597:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72310784 chunk: [72310784, 72314880) 72314880, 4096 00020000:00200000:1.0:1789760782.382602:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.382624:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17654, 17654] 00000008:00010000:1.0:1789760782.382808:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.382815:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.382827:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72310784, 72314880) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.382842:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72310784=0x44f6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.382909:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.382912:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72314880 00000020:00010000:1.0:1789760782.382920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.382923:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.382927:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.382930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.382933:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.382941:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.382948:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.382968:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760782.383212:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 72310784<>72314880 00000100:00100000:0.0:1789760782.383284:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565490176:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.405788:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984962 gen 3 00000008:00010000:0.0:1789760782.405844:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760782.405854:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.405866:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.405888:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760782.405920:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565490176:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.406155:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17654, 17654] 00000080:00200000:1.0:1789760782.406173:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.406180:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.406197:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72314880 i_blocks 141264 00000020:00200000:1.0:1789760782.406201:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.406211:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.406249:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72314880, count: 4096 00000080:00200000:1.0:1789760782.406256:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72314880, bytes: 4096 00000080:00200000:1.0:1789760782.406267:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.406276:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.406279:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72314880, 72318976) 0 0 00000080:00200000:1.0:1789760782.406284:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.406305:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17655, 17655] 00020000:00200000:1.0:1789760782.406314:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72314880 chunk: [72314880, 72318976) 72318976, 4096 00020000:00200000:1.0:1789760782.406320:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.406342:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17655, 17655] 00000008:00010000:1.0:1789760782.406362:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.406368:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.406381:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72314880, 72318976) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.406396:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72314880=0x44f7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.406461:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.406464:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72318976 00000020:00010000:1.0:1789760782.406472:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.406476:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.406491:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.406496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.406498:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.406506:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.406513:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.406532:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760782.407015:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 72314880<>72318976 00000100:00100000:0.0:1789760782.407352:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565490304:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.420241:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984963 gen 3 00000008:00010000:0.0:1789760782.420294:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760782.420303:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.420315:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.420338:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760782.420368:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565490304:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.420487:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17655, 17655] 00000080:00200000:1.0:1789760782.420504:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.420510:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.420526:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72318976 i_blocks 141272 00000020:00200000:1.0:1789760782.420530:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.420539:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.420585:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72318976, count: 4096 00000080:00200000:1.0:1789760782.420593:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72318976, bytes: 4096 00000080:00200000:1.0:1789760782.420603:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.420611:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.420614:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72318976, 72323072) 0 0 00000080:00200000:1.0:1789760782.420619:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.420861:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17656, 17656] 00020000:00200000:1.0:1789760782.420871:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72318976 chunk: [72318976, 72323072) 72323072, 4096 00020000:00200000:1.0:1789760782.420877:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.420897:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17656, 17656] 00000008:00010000:1.0:1789760782.420915:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.420921:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.420932:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72318976, 72323072) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.420947:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72318976=0x44f8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.421004:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.421007:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72323072 00000020:00010000:1.0:1789760782.421015:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.421019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.421022:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.421025:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0e48 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.421028:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.422532:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.422549:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.422585:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760782.423094:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 72318976<>72323072 00000100:00100000:0.0:1789760782.423172:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565490432:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.434696:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984964 gen 3 00000008:00010000:0.0:1789760782.434753:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760782.434772:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.434784:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.434806:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760782.434837:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565490432:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760782.434927:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17656, 17656] 00000080:00200000:3.0:1789760782.434945:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760782.434952:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760782.434968:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72323072 i_blocks 141280 00000020:00200000:3.0:1789760782.434972:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760782.434982:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760782.435051:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72323072, count: 4096 00000080:00200000:3.0:1789760782.435059:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72323072, bytes: 4096 00000080:00200000:3.0:1789760782.435071:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760782.435080:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760782.435082:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72323072, 72327168) 0 0 00000080:00200000:3.0:1789760782.435086:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760782.435106:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17657, 17657] 00020000:00200000:3.0:1789760782.435115:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72323072 chunk: [72323072, 72327168) 72327168, 4096 00020000:00200000:3.0:1789760782.435119:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760782.435139:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17657, 17657] 00000008:00010000:3.0:1789760782.435158:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760782.435165:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760782.435176:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72323072, 72327168) append 0 append lockpos 0 00000080:00200000:3.0:1789760782.435190:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72323072=0x44f9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760782.435248:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760782.435251:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72327168 00000020:00010000:3.0:1789760782.435261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760782.435264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760782.435267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760782.435270:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760782.435274:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760782.435282:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760782.435289:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760782.435306:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760782.435441:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 72323072<>72327168 00000100:00100000:3.0:1789760782.435598:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760782.435618:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565490560:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.443065:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984965 gen 3 00000008:00010000:3.0:1789760782.443119:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:3.0:1789760782.443127:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.443138:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.443162:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:3.0:1789760782.443191:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565490560:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.443319:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17657, 17657] 00000080:00200000:2.0:1789760782.443337:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.443343:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.443358:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72327168 i_blocks 141288 00000020:00200000:2.0:1789760782.443362:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.443373:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.443407:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72327168, count: 4096 00000080:00200000:2.0:1789760782.443414:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72327168, bytes: 4096 00000080:00200000:2.0:1789760782.443436:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.443447:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.443449:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72327168, 72331264) 0 0 00000080:00200000:2.0:1789760782.443454:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.443471:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17658, 17658] 00020000:00200000:2.0:1789760782.443496:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72327168 chunk: [72327168, 72331264) 72331264, 4096 00020000:00200000:2.0:1789760782.443501:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.443522:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17658, 17658] 00000008:00010000:2.0:1789760782.443539:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.443547:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.443700:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72327168, 72331264) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.443716:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72327168=0x44fa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.443778:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.443781:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72331264 00000020:00010000:2.0:1789760782.443790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.443794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.443797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.443800:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.443803:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.443811:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.443819:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.443845:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760782.443969:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 72327168<>72331264 00000100:00100000:2.0:1789760782.444016:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760782.444035:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565490688:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.454294:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984966 gen 3 00000008:00010000:2.0:1789760782.454346:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760782.454355:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.454366:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.454564:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760782.454597:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565490688:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.455115:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17658, 17658] 00000080:00200000:0.0:1789760782.455134:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.455140:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.455156:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72331264 i_blocks 141296 00000020:00200000:0.0:1789760782.455161:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.455172:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.455206:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72331264, count: 4096 00000080:00200000:0.0:1789760782.455213:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72331264, bytes: 4096 00000080:00200000:0.0:1789760782.455225:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.455234:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.455237:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72331264, 72335360) 0 0 00000080:00200000:0.0:1789760782.455241:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.455257:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17659, 17659] 00020000:00200000:0.0:1789760782.455266:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72331264 chunk: [72331264, 72335360) 72335360, 4096 00020000:00200000:0.0:1789760782.455271:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.455290:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17659, 17659] 00000008:00010000:0.0:1789760782.455307:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.455313:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.455325:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72331264, 72335360) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.455341:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72331264=0x44fb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.455403:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.455406:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72335360 00000020:00010000:0.0:1789760782.455530:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.455535:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.455539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.455542:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.455545:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.455593:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.455601:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.455624:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.455779:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72331264<>72335360 00000100:00100000:0.0:1789760782.455816:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760782.455931:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565490816:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.465984:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984967 gen 3 00000008:00010000:0.0:1789760782.466041:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.466050:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.466061:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.466089:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.466120:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565490816:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.466363:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17659, 17659] 00000080:00200000:2.0:1789760782.466381:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.466387:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.466402:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72335360 i_blocks 141304 00000020:00200000:2.0:1789760782.466406:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.466417:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.466452:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72335360, count: 4096 00000080:00200000:2.0:1789760782.466459:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72335360, bytes: 4096 00000080:00200000:2.0:1789760782.466468:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.466488:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.466491:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72335360, 72339456) 0 0 00000080:00200000:2.0:1789760782.466496:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.466513:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17660, 17660] 00020000:00200000:2.0:1789760782.466521:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72335360 chunk: [72335360, 72339456) 72339456, 4096 00020000:00200000:2.0:1789760782.466525:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.466543:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17660, 17660] 00000008:00010000:2.0:1789760782.466559:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.466566:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.466577:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72335360, 72339456) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.466590:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72335360=0x44fc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.466692:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.466695:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72339456 00000020:00010000:2.0:1789760782.466702:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.466706:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.466709:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.466713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.466716:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.466723:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.466728:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.466744:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760782.467536:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 72335360<>72339456 00000100:00100000:3.0:1789760782.467623:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760782.467678:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565490944:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.478966:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984968 gen 3 00000008:00010000:3.0:1789760782.479021:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760782.479030:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.479041:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.479065:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760782.479097:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565490944:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.479497:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17660, 17660] 00000080:00200000:2.0:1789760782.479520:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.479526:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.479546:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72339456 i_blocks 141312 00000020:00200000:2.0:1789760782.479555:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.479567:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.479611:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72339456, count: 4096 00000080:00200000:2.0:1789760782.479621:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72339456, bytes: 4096 00000080:00200000:2.0:1789760782.479632:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.479682:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.479685:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72339456, 72343552) 0 0 00000080:00200000:2.0:1789760782.479689:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.479710:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17661, 17661] 00020000:00200000:2.0:1789760782.479721:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72339456 chunk: [72339456, 72343552) 72343552, 4096 00020000:00200000:2.0:1789760782.479726:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.479750:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17661, 17661] 00000008:00010000:2.0:1789760782.479773:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.479780:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.479803:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72339456, 72343552) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.479820:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72339456=0x44fd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.479890:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.479893:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72343552 00000020:00010000:2.0:1789760782.479903:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.479907:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.479910:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.479914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.479917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.479925:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.479933:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.479952:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.480092:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 72339456<>72343552 00000100:00100000:2.0:1789760782.480138:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760782.480158:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565491072:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.489270:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984969 gen 3 00000008:00010000:2.0:1789760782.489326:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760782.489336:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.489350:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.489701:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760782.489731:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565491072:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.490262:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17661, 17661] 00000080:00200000:0.0:1789760782.490281:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.490289:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.490306:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72343552 i_blocks 141320 00000020:00200000:0.0:1789760782.490310:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.490321:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.490357:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72343552, count: 4096 00000080:00200000:0.0:1789760782.490365:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72343552, bytes: 4096 00000080:00200000:0.0:1789760782.490378:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.490387:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.490389:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72343552, 72347648) 0 0 00000080:00200000:0.0:1789760782.490393:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.490410:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17662, 17662] 00020000:00200000:0.0:1789760782.490548:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72343552 chunk: [72343552, 72347648) 72347648, 4096 00020000:00200000:0.0:1789760782.490597:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.490619:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17662, 17662] 00000008:00010000:0.0:1789760782.490638:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.490663:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.490676:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72343552, 72347648) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.490691:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72343552=0x44fe000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.490759:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.490762:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72347648 00000020:00010000:0.0:1789760782.490771:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.490775:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.490777:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.490953:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.490957:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.490968:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.490975:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.491001:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.491260:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72343552<>72347648 00000100:00100000:1.0:1789760782.491749:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565491200:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.502186:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984970 gen 3 00000008:00010000:0.0:1789760782.502357:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.502366:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.502376:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.502458:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.502504:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565491200:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.503324:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17662, 17662] 00000080:00200000:2.0:1789760782.503344:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.503350:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.503366:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72347648 i_blocks 141328 00000020:00200000:2.0:1789760782.503370:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.503384:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.503430:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72347648, count: 4096 00000080:00200000:2.0:1789760782.503438:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72347648, bytes: 4096 00000080:00200000:2.0:1789760782.503451:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.503460:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.503463:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72347648, 72351744) 0 0 00000080:00200000:2.0:1789760782.503467:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.503494:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17663, 17663] 00020000:00200000:2.0:1789760782.503503:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72347648 chunk: [72347648, 72351744) 72351744, 4096 00020000:00200000:2.0:1789760782.503508:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.503529:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17663, 17663] 00000008:00010000:2.0:1789760782.503548:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.503555:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.503567:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72347648, 72351744) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.503581:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72347648=0x44ff000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.503661:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.503664:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72351744 00000020:00010000:2.0:1789760782.503672:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.503676:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.503679:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.503683:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.503686:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.503695:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.503701:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.503723:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.503866:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 72347648<>72351744 00000100:00100000:3.0:1789760782.504159:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565491328:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.513005:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984971 gen 3 00000008:00010000:3.0:1789760782.513068:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760782.513077:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.513089:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.513169:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760782.513209:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565491328:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.517289:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17663, 17663] 00000080:00200000:2.0:1789760782.517318:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.517325:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.517345:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72351744 i_blocks 141336 00000020:00200000:2.0:1789760782.517350:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.517365:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.517401:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72351744, count: 4096 00000080:00200000:2.0:1789760782.517411:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72351744, bytes: 4096 00000080:00200000:2.0:1789760782.517425:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.517434:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.517437:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72351744, 72355840) 0 0 00000080:00200000:2.0:1789760782.517441:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.517458:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17664, 17664] 00020000:00200000:2.0:1789760782.517468:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72351744 chunk: [72351744, 72355840) 72355840, 4096 00020000:00200000:2.0:1789760782.517472:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.517539:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17664, 17664] 00000008:00010000:2.0:1789760782.517763:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.517771:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.518617:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72351744, 72355840) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.518732:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72351744=0x4500000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.518925:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.518929:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72355840 00000020:00010000:2.0:1789760782.518941:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.518945:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.519004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.519008:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.519011:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.519025:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.519098:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.519133:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.519277:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 72351744<>72355840 00000100:00100000:3.0:1789760782.519381:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565491456:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.533927:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984972 gen 3 00000008:00010000:3.0:1789760782.533976:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760782.533985:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.533997:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.534020:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760782.534050:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565491456:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.534168:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17664, 17664] 00000080:00200000:2.0:1789760782.534185:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.534255:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.536010:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72355840 i_blocks 141344 00000020:00200000:2.0:1789760782.536026:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.536176:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.536275:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72355840, count: 4096 00000080:00200000:2.0:1789760782.536285:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72355840, bytes: 4096 00000080:00200000:2.0:1789760782.536299:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.536308:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.536311:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72355840, 72359936) 0 0 00000080:00200000:2.0:1789760782.536316:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.536334:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17665, 17665] 00020000:00200000:2.0:1789760782.536344:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72355840 chunk: [72355840, 72359936) 72359936, 4096 00020000:00200000:2.0:1789760782.536349:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.536372:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17665, 17665] 00000008:00010000:2.0:1789760782.536391:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.536397:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.536409:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72355840, 72359936) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.536423:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72355840=0x4501000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.536610:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.536613:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72359936 00000020:00010000:2.0:1789760782.536622:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.536626:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.536629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.536633:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.536635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.536664:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.536671:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.536692:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760782.536820:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72355840<>72359936 00000100:00100000:3.0:1789760782.537073:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565491584:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.552661:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984973 gen 3 00000008:00010000:3.0:1789760782.552712:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760782.552721:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.552732:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.552758:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760782.552791:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565491584:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.553003:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17665, 17665] 00000080:00200000:2.0:1789760782.553081:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.553088:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.553104:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72359936 i_blocks 141352 00000020:00200000:2.0:1789760782.553108:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.553117:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.553150:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72359936, count: 4096 00000080:00200000:2.0:1789760782.553157:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72359936, bytes: 4096 00000080:00200000:2.0:1789760782.553169:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.553177:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.553180:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72359936, 72364032) 0 0 00000080:00200000:2.0:1789760782.553185:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.553201:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17666, 17666] 00020000:00200000:2.0:1789760782.553210:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72359936 chunk: [72359936, 72364032) 72364032, 4096 00020000:00200000:2.0:1789760782.553215:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.553234:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17666, 17666] 00000008:00010000:2.0:1789760782.553250:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.553256:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.553268:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72359936, 72364032) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.553281:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72359936=0x4502000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.553338:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.553342:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72364032 00000020:00010000:2.0:1789760782.553349:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.553353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.553356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.553359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.553362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.553370:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.553377:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.553395:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760782.553527:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 72359936<>72364032 00000100:00100000:3.0:1789760782.553715:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760782.553732:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565491712:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.565238:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984974 gen 3 00000008:00010000:3.0:1789760782.565291:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760782.565301:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.565311:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.565336:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760782.565367:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565491712:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.565570:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17666, 17666] 00000080:00200000:2.0:1789760782.565587:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.565593:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.565607:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72364032 i_blocks 141360 00000020:00200000:2.0:1789760782.565611:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.565623:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.565693:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72364032, count: 4096 00000080:00200000:2.0:1789760782.565701:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72364032, bytes: 4096 00000080:00200000:2.0:1789760782.565712:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.565721:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.565723:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72364032, 72368128) 0 0 00000080:00200000:2.0:1789760782.565728:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.565744:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17667, 17667] 00020000:00200000:2.0:1789760782.565754:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72364032 chunk: [72364032, 72368128) 72368128, 4096 00020000:00200000:2.0:1789760782.565759:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.565780:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17667, 17667] 00000008:00010000:2.0:1789760782.565797:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.565803:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.565816:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72364032, 72368128) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.566052:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72364032=0x4503000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.566112:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.566115:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72368128 00000020:00010000:2.0:1789760782.566123:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.566127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.566130:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.566134:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.566137:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.566146:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.566153:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.566233:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760782.566355:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 72364032<>72368128 00000100:00100000:3.0:1789760782.566537:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565491840:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.576248:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984975 gen 3 00000008:00010000:2.0:1789760782.576301:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760782.576310:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.576322:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.576338:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760782.576368:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565491840:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.576702:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17667, 17667] 00000080:00200000:0.0:1789760782.576719:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.576725:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.576741:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72368128 i_blocks 141368 00000020:00200000:0.0:1789760782.576745:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.576755:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.576787:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72368128, count: 4096 00000080:00200000:0.0:1789760782.576794:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72368128, bytes: 4096 00000080:00200000:0.0:1789760782.576804:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.576813:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.576816:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72368128, 72372224) 0 0 00000080:00200000:0.0:1789760782.576821:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.576837:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17668, 17668] 00020000:00200000:0.0:1789760782.576846:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72368128 chunk: [72368128, 72372224) 72372224, 4096 00020000:00200000:0.0:1789760782.576851:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.576870:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17668, 17668] 00000008:00010000:0.0:1789760782.576889:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.576896:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.576906:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72368128, 72372224) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.576920:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72368128=0x4504000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.577051:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.577054:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72372224 00000020:00010000:0.0:1789760782.577063:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.577067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.577070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.577074:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.577077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.577085:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.577093:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.577109:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.577910:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72368128<>72372224 00000100:00100000:1.0:1789760782.577970:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565491968:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.590146:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984976 gen 3 00000008:00010000:0.0:1789760782.590217:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.590227:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.590239:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.590261:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.590290:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565491968:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.590460:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17668, 17668] 00000080:00200000:2.0:1789760782.590581:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.590588:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.590603:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72372224 i_blocks 141376 00000020:00200000:2.0:1789760782.590607:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.590618:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.590687:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72372224, count: 4096 00000080:00200000:2.0:1789760782.590695:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72372224, bytes: 4096 00000080:00200000:2.0:1789760782.590708:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.590717:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.590719:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72372224, 72376320) 0 0 00000080:00200000:2.0:1789760782.590724:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.590740:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17669, 17669] 00020000:00200000:2.0:1789760782.590751:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72372224 chunk: [72372224, 72376320) 72376320, 4096 00020000:00200000:2.0:1789760782.590755:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.590776:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17669, 17669] 00000008:00010000:2.0:1789760782.590795:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.590801:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.590812:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72372224, 72376320) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.590826:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72372224=0x4505000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.590887:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.590891:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72376320 00000020:00010000:2.0:1789760782.590899:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.590903:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.590906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.590909:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.590912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.590920:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.590926:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.590941:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760782.591586:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 72372224<>72376320 00000100:00100000:2.0:1789760782.591666:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565492096:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.607622:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984977 gen 3 00000008:00010000:2.0:1789760782.607717:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760782.607726:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.607737:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.607761:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760782.607789:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565492096:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760782.608252:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17669, 17669] 00000080:00200000:3.0:1789760782.608270:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760782.608276:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760782.608292:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72376320 i_blocks 141384 00000020:00200000:3.0:1789760782.608295:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760782.608305:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760782.608338:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72376320, count: 4096 00000080:00200000:3.0:1789760782.608344:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72376320, bytes: 4096 00000080:00200000:3.0:1789760782.608355:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760782.608365:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760782.608367:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72376320, 72380416) 0 0 00000080:00200000:3.0:1789760782.608372:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760782.608389:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17670, 17670] 00020000:00200000:3.0:1789760782.608398:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72376320 chunk: [72376320, 72380416) 72380416, 4096 00020000:00200000:3.0:1789760782.608402:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760782.608431:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17670, 17670] 00000008:00010000:3.0:1789760782.608448:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760782.608454:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760782.608464:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72376320, 72380416) append 0 append lockpos 0 00000080:00200000:3.0:1789760782.608492:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72376320=0x4506000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760782.608547:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760782.608549:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72380416 00000020:00010000:3.0:1789760782.608557:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760782.608560:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760782.608563:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760782.608575:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760782.608578:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760782.608587:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760782.608593:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760782.608608:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760782.608743:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 72376320<>72380416 00000100:00100000:2.0:1789760782.608819:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565492224:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.621350:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984978 gen 3 00000008:00010000:2.0:1789760782.621407:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760782.621416:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.621427:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.621457:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760782.621494:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565492224:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760782.621611:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17670, 17670] 00000080:00200000:3.0:1789760782.621630:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760782.621636:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760782.621681:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72380416 i_blocks 141392 00000020:00200000:3.0:1789760782.621686:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760782.621710:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760782.621745:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72380416, count: 4096 00000080:00200000:3.0:1789760782.621753:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72380416, bytes: 4096 00000080:00200000:3.0:1789760782.621763:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760782.621771:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760782.621773:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72380416, 72384512) 0 0 00000080:00200000:3.0:1789760782.621778:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760782.621794:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17671, 17671] 00020000:00200000:3.0:1789760782.621802:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72380416 chunk: [72380416, 72384512) 72384512, 4096 00020000:00200000:3.0:1789760782.621807:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760782.621826:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17671, 17671] 00000008:00010000:3.0:1789760782.621843:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760782.621849:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760782.621861:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72380416, 72384512) append 0 append lockpos 0 00000080:00200000:3.0:1789760782.621874:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72380416=0x4507000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760782.621924:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760782.621927:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72384512 00000020:00010000:3.0:1789760782.621935:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760782.621939:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760782.621942:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760782.621946:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760782.621949:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760782.621959:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760782.621966:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760782.621980:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760782.622113:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 72380416<>72384512 00000100:00100000:2.0:1789760782.622197:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565492352:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.629853:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984979 gen 3 00000008:00010000:2.0:1789760782.629916:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760782.629925:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.629936:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.629963:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760782.629999:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565492352:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760782.630537:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17671, 17671] 00000080:00200000:3.0:1789760782.630556:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760782.630562:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760782.630577:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72384512 i_blocks 141400 00000020:00200000:3.0:1789760782.630582:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760782.630591:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760782.630624:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72384512, count: 4096 00000080:00200000:3.0:1789760782.630633:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72384512, bytes: 4096 00000080:00200000:3.0:1789760782.630692:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760782.630701:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760782.630704:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72384512, 72388608) 0 0 00000080:00200000:3.0:1789760782.630709:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760782.630727:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17672, 17672] 00020000:00200000:3.0:1789760782.630737:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72384512 chunk: [72384512, 72388608) 72388608, 4096 00020000:00200000:3.0:1789760782.630742:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760782.630763:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17672, 17672] 00000008:00010000:3.0:1789760782.630783:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760782.630788:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760782.630800:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72384512, 72388608) append 0 append lockpos 0 00000080:00200000:3.0:1789760782.630814:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72384512=0x4508000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760782.630878:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760782.630881:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72388608 00000020:00010000:3.0:1789760782.630890:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760782.630893:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760782.630897:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760782.630900:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760782.630903:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760782.630915:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760782.630922:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.630955:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:2.0:1789760782.631191:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 72384512<>72388608 00000100:00100000:2.0:1789760782.631264:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565492480:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.644280:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984980 gen 3 00000008:00010000:2.0:1789760782.644329:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760782.644338:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.644349:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.644368:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760782.644394:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565492480:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760782.644960:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17672, 17672] 00000080:00200000:3.0:1789760782.644989:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760782.644995:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760782.645012:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72388608 i_blocks 141408 00000020:00200000:3.0:1789760782.645016:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760782.645026:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760782.645061:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72388608, count: 4096 00000080:00200000:3.0:1789760782.645070:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72388608, bytes: 4096 00000080:00200000:3.0:1789760782.645082:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760782.645091:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760782.645094:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72388608, 72392704) 0 0 00000080:00200000:3.0:1789760782.645098:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760782.645117:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17673, 17673] 00020000:00200000:3.0:1789760782.645126:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72388608 chunk: [72388608, 72392704) 72392704, 4096 00020000:00200000:3.0:1789760782.645132:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760782.645153:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17673, 17673] 00000008:00010000:3.0:1789760782.645172:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760782.645178:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760782.645190:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72388608, 72392704) append 0 append lockpos 0 00000080:00200000:3.0:1789760782.645204:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72388608=0x4509000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760782.645265:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760782.645268:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72392704 00000020:00010000:3.0:1789760782.645275:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760782.645278:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760782.645281:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760782.645285:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760782.645287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760782.645297:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760782.645304:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760782.645323:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760782.645450:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 72388608<>72392704 00000100:00100000:2.0:1789760782.645543:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565492608:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.663105:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984981 gen 3 00000008:00010000:2.0:1789760782.663155:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760782.663164:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.663176:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.663266:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760782.663297:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565492608:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760782.663733:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17673, 17673] 00000080:00200000:3.0:1789760782.663748:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760782.663754:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.664051:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72392704 i_blocks 141416 00000020:00200000:2.0:1789760782.664056:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.664067:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.664101:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72392704, count: 4096 00000080:00200000:2.0:1789760782.664108:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72392704, bytes: 4096 00000080:00200000:2.0:1789760782.664118:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.664127:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.664129:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72392704, 72396800) 0 0 00000080:00200000:2.0:1789760782.664134:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.664150:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17674, 17674] 00020000:00200000:2.0:1789760782.664158:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72392704 chunk: [72392704, 72396800) 72396800, 4096 00020000:00200000:2.0:1789760782.664163:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.664180:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17674, 17674] 00000008:00010000:2.0:1789760782.664198:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.664203:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.664214:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72392704, 72396800) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.664226:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72392704=0x450a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.664281:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.664284:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72396800 00000020:00010000:2.0:1789760782.664292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.664296:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.664299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.664302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.664305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.664313:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.664320:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.664368:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760782.664613:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 72392704<>72396800 00000100:00100000:3.0:1789760782.664687:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565492736:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.676514:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984982 gen 3 00000008:00010000:2.0:1789760782.676565:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760782.676573:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.676584:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.676611:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760782.676703:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565492736:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.676850:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17674, 17674] 00000080:00200000:0.0:1789760782.676866:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.676872:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.676886:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72396800 i_blocks 141424 00000020:00200000:0.0:1789760782.676890:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.676900:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.676930:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72396800, count: 4096 00000080:00200000:0.0:1789760782.676949:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72396800, bytes: 4096 00000080:00200000:0.0:1789760782.676961:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.676970:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.676972:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72396800, 72400896) 0 0 00000080:00200000:0.0:1789760782.676976:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.676994:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17675, 17675] 00020000:00200000:0.0:1789760782.677002:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72396800 chunk: [72396800, 72400896) 72400896, 4096 00020000:00200000:0.0:1789760782.677007:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.677027:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17675, 17675] 00000008:00010000:0.0:1789760782.677048:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.677059:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.677072:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72396800, 72400896) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.677086:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72396800=0x450b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.677145:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.677148:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72400896 00000020:00010000:0.0:1789760782.677157:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.677161:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.677164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.677168:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.677171:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.677179:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.677186:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.677201:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.677340:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72396800<>72400896 00000100:00100000:0.0:1789760782.677396:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760782.677413:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565492864:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.687267:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984983 gen 3 00000008:00010000:0.0:1789760782.687318:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.687328:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.687341:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.687418:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.687448:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565492864:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.688024:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17675, 17675] 00000080:00200000:1.0:1789760782.688042:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.688048:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.688066:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72400896 i_blocks 141432 00000020:00200000:1.0:1789760782.688070:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.688082:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.688117:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72400896, count: 4096 00000080:00200000:1.0:1789760782.688124:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72400896, bytes: 4096 00000080:00200000:1.0:1789760782.688137:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.688145:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.688148:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72400896, 72404992) 0 0 00000080:00200000:1.0:1789760782.688152:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.688169:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17676, 17676] 00020000:00200000:1.0:1789760782.688178:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72400896 chunk: [72400896, 72404992) 72404992, 4096 00020000:00200000:1.0:1789760782.688184:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.688205:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17676, 17676] 00000008:00010000:1.0:1789760782.688223:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.688229:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.688241:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72400896, 72404992) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.688254:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72400896=0x450c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.688314:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.688317:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72404992 00000020:00010000:1.0:1789760782.688326:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.688329:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.688332:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.688335:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.688339:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.688346:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.688353:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.688373:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760782.688811:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 72400896<>72404992 00000100:00100000:1.0:1789760782.688900:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760782.689028:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565492992:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.701703:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984984 gen 3 00000008:00010000:0.0:1789760782.701758:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760782.701767:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.701778:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.701983:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760782.702016:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565492992:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.702358:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17676, 17676] 00000080:00200000:2.0:1789760782.702375:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.702382:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.702396:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72404992 i_blocks 141440 00000020:00200000:2.0:1789760782.702400:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.702411:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.702973:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72404992, count: 4096 00000080:00200000:2.0:1789760782.702984:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72404992, bytes: 4096 00000080:00200000:2.0:1789760782.702997:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.703006:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.703008:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72404992, 72409088) 0 0 00000080:00200000:2.0:1789760782.703014:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.703032:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17677, 17677] 00020000:00200000:2.0:1789760782.703043:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72404992 chunk: [72404992, 72409088) 72409088, 4096 00020000:00200000:2.0:1789760782.703049:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.703071:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17677, 17677] 00000008:00010000:2.0:1789760782.703092:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.703100:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.703113:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72404992, 72409088) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.703129:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72404992=0x450d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.703191:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.703195:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72409088 00000020:00010000:2.0:1789760782.703202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.703206:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.703209:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.703212:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.703214:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.703223:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.703229:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.703251:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760782.703377:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 72404992<>72409088 00000100:00100000:3.0:1789760782.703474:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565493120:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.713281:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984985 gen 3 00000008:00010000:3.0:1789760782.713338:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760782.713346:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.713359:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.713455:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760782.713504:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565493120:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.713970:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17677, 17677] 00000080:00200000:0.0:1789760782.713986:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.713993:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.714009:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72409088 i_blocks 141448 00000020:00200000:0.0:1789760782.714013:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.714024:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.714059:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72409088, count: 4096 00000080:00200000:0.0:1789760782.714066:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72409088, bytes: 4096 00000080:00200000:0.0:1789760782.714079:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.714088:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.714091:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72409088, 72413184) 0 0 00000080:00200000:0.0:1789760782.714095:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.714112:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17678, 17678] 00020000:00200000:0.0:1789760782.714121:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72409088 chunk: [72409088, 72413184) 72413184, 4096 00020000:00200000:0.0:1789760782.714126:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.714144:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17678, 17678] 00000008:00010000:0.0:1789760782.714160:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.714167:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.714180:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72409088, 72413184) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.714196:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72409088=0x450e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.714251:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.714255:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72413184 00000020:00010000:0.0:1789760782.714262:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.714266:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.714269:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.714273:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.714276:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.714283:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.714290:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.714310:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.714568:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72409088<>72413184 00000100:00100000:1.0:1789760782.715004:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760782.715088:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565493248:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.724932:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984986 gen 3 00000008:00010000:0.0:1789760782.724988:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.724997:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.725006:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.725028:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.725060:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565493248:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.725227:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17678, 17678] 00000080:00200000:1.0:1789760782.725245:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.725251:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.725268:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72413184 i_blocks 141456 00000020:00200000:1.0:1789760782.725272:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.725284:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.725320:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72413184, count: 4096 00000080:00200000:1.0:1789760782.725329:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72413184, bytes: 4096 00000080:00200000:1.0:1789760782.725341:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.725352:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.725355:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72413184, 72417280) 0 0 00000080:00200000:1.0:1789760782.725360:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.725379:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17679, 17679] 00020000:00200000:1.0:1789760782.725389:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72413184 chunk: [72413184, 72417280) 72417280, 4096 00020000:00200000:1.0:1789760782.725394:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.725416:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17679, 17679] 00000008:00010000:1.0:1789760782.725435:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.725442:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.725454:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72413184, 72417280) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.725470:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72413184=0x450f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.725551:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.725554:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72417280 00000020:00010000:1.0:1789760782.725564:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.725567:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.725571:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.725574:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.725577:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.725595:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.725610:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.725629:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760782.726141:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 72413184<>72417280 00000100:00100000:0.0:1789760782.726299:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760782.726322:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565493376:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.736496:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984987 gen 3 00000008:00010000:0.0:1789760782.736549:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760782.736558:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.736569:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.736884:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760782.736930:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565493376:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.737559:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17679, 17679] 00000080:00200000:1.0:1789760782.737578:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.737742:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.737762:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72417280 i_blocks 141464 00000020:00200000:1.0:1789760782.737767:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.737782:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.737817:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72417280, count: 4096 00000080:00200000:1.0:1789760782.737825:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72417280, bytes: 4096 00000080:00200000:1.0:1789760782.737838:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.737848:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.737851:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72417280, 72421376) 0 0 00000080:00200000:1.0:1789760782.737855:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.737872:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17680, 17680] 00020000:00200000:1.0:1789760782.737881:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72417280 chunk: [72417280, 72421376) 72421376, 4096 00020000:00200000:1.0:1789760782.737886:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.739023:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17680, 17680] 00000008:00010000:1.0:1789760782.739059:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.739065:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.739079:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72417280, 72421376) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.739094:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72417280=0x4510000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.739167:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.739170:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72421376 00000020:00010000:1.0:1789760782.739178:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.739181:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.739184:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.739188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.739191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.739201:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.739209:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.739233:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760782.739361:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 72417280<>72421376 00000100:00100000:0.0:1789760782.739659:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565493504:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.748877:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984988 gen 3 00000008:00010000:0.0:1789760782.749109:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760782.749118:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.749128:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.749195:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760782.749227:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565493504:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.749374:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17680, 17680] 00000080:00200000:1.0:1789760782.749392:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.749397:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.749413:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72421376 i_blocks 141472 00000020:00200000:1.0:1789760782.749418:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.749427:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.749462:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72421376, count: 4096 00000080:00200000:1.0:1789760782.749469:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72421376, bytes: 4096 00000080:00200000:1.0:1789760782.749487:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.749497:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.749499:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72421376, 72425472) 0 0 00000080:00200000:1.0:1789760782.749504:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.749524:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17681, 17681] 00020000:00200000:1.0:1789760782.749533:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72421376 chunk: [72421376, 72425472) 72425472, 4096 00020000:00200000:1.0:1789760782.749538:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.749563:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17681, 17681] 00000008:00010000:1.0:1789760782.749672:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.749734:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.749750:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72421376, 72425472) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.749764:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72421376=0x4511000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.749830:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.749833:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72425472 00000020:00010000:1.0:1789760782.749842:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.749845:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.749848:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.749852:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.749855:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.749862:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.749870:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.749889:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760782.750211:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 72421376<>72425472 00000100:00100000:0.0:1789760782.750445:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565493632:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.761064:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984989 gen 3 00000008:00010000:0.0:1789760782.761116:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760782.761124:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.761136:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.761158:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760782.761187:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565493632:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760782.761304:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17681, 17681] 00000080:00200000:1.0:1789760782.761321:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760782.761327:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760782.761342:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72425472 i_blocks 141480 00000020:00200000:1.0:1789760782.761346:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760782.761364:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760782.761398:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72425472, count: 4096 00000080:00200000:1.0:1789760782.761405:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72425472, bytes: 4096 00000080:00200000:1.0:1789760782.761415:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760782.761424:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760782.761427:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72425472, 72429568) 0 0 00000080:00200000:1.0:1789760782.761431:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760782.761448:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17682, 17682] 00020000:00200000:1.0:1789760782.761457:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72425472 chunk: [72425472, 72429568) 72429568, 4096 00020000:00200000:1.0:1789760782.761461:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760782.761490:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17682, 17682] 00000008:00010000:1.0:1789760782.761509:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760782.761515:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760782.761527:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72425472, 72429568) append 0 append lockpos 0 00000080:00200000:1.0:1789760782.761542:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72425472=0x4512000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760782.761600:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760782.761603:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72429568 00000020:00010000:1.0:1789760782.761611:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760782.761614:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.761616:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760782.761620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760782.761623:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760782.761630:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760782.761636:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760782.761697:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760782.762001:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 72425472<>72429568 00000100:00100000:0.0:1789760782.762058:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565493760:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.773231:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984990 gen 3 00000008:00010000:0.0:1789760782.773281:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760782.773290:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.773300:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.773330:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000080:00200000:2.0:1789760782.773366:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17682, 17682] 00000100:00100000:0.0:1789760782.773371:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565493760:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.773386:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.773393:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.773408:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72429568 i_blocks 141488 00000020:00200000:2.0:1789760782.773411:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.773423:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.773458:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72429568, count: 4096 00000080:00200000:2.0:1789760782.773465:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72429568, bytes: 4096 00000080:00200000:2.0:1789760782.773475:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.773496:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.773499:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72429568, 72433664) 0 0 00000080:00200000:2.0:1789760782.773503:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.773522:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17683, 17683] 00020000:00200000:2.0:1789760782.773530:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72429568 chunk: [72429568, 72433664) 72433664, 4096 00020000:00200000:2.0:1789760782.773535:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.773556:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17683, 17683] 00000008:00010000:2.0:1789760782.773573:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.773579:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.773591:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72429568, 72433664) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.773604:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72429568=0x4513000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.773696:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.773699:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72433664 00000020:00010000:2.0:1789760782.773707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.773710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.773713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.773717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.773720:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.773728:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.773735:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.773751:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760782.773873:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 72429568<>72433664 00000100:00100000:3.0:1789760782.773999:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565493888:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.782457:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984991 gen 3 00000008:00010000:3.0:1789760782.782518:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760782.782528:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.782539:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.782565:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760782.782596:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565493888:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.782889:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17683, 17683] 00000080:00200000:2.0:1789760782.782906:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.782912:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.782927:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72433664 i_blocks 141496 00000020:00200000:2.0:1789760782.782931:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.782940:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.782975:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72433664, count: 4096 00000080:00200000:2.0:1789760782.782983:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72433664, bytes: 4096 00000080:00200000:2.0:1789760782.782992:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.783001:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.783003:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72433664, 72437760) 0 0 00000080:00200000:2.0:1789760782.783008:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.783024:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17684, 17684] 00020000:00200000:2.0:1789760782.783034:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72433664 chunk: [72433664, 72437760) 72437760, 4096 00020000:00200000:2.0:1789760782.783039:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.783060:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17684, 17684] 00000008:00010000:2.0:1789760782.783078:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.783084:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.783096:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72433664, 72437760) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.783110:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72433664=0x4514000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.783173:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.783177:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72437760 00000020:00010000:2.0:1789760782.783186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.783190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.783193:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.783197:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.783201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.783209:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.783217:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.783234:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.783366:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 72433664<>72437760 00000100:00100000:3.0:1789760782.783443:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565494016:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760782.802244:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984992 gen 3 00000008:00010000:3.0:1789760782.802296:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760782.802305:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760782.802316:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760782.802342:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760782.802370:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565494016:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.802523:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17684, 17684] 00000080:00200000:0.0:1789760782.802540:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.802546:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.802562:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72437760 i_blocks 141504 00000020:00200000:0.0:1789760782.802565:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.802576:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.802610:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72437760, count: 4096 00000080:00200000:0.0:1789760782.802617:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72437760, bytes: 4096 00000080:00200000:0.0:1789760782.802632:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.802641:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.802676:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72437760, 72441856) 0 0 00000080:00200000:0.0:1789760782.802681:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.802708:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17685, 17685] 00020000:00200000:0.0:1789760782.802717:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72437760 chunk: [72437760, 72441856) 72441856, 4096 00020000:00200000:0.0:1789760782.802723:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.802742:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17685, 17685] 00000008:00010000:0.0:1789760782.802759:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.802766:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.802778:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72437760, 72441856) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.802791:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72437760=0x4515000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.803127:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.803131:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72441856 00000020:00010000:0.0:1789760782.803140:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.803144:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.803147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.803151:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.803154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.803162:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.803169:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.803186:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.803307:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72437760<>72441856 00000100:00100000:1.0:1789760782.803378:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565494144:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.814535:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984993 gen 3 00000008:00010000:0.0:1789760782.814585:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.814593:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.814604:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.814632:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.814693:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565494144:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.814877:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17685, 17685] 00000080:00200000:2.0:1789760782.814896:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.814902:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.814916:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72441856 i_blocks 141512 00000020:00200000:2.0:1789760782.814919:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.814930:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.814964:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72441856, count: 4096 00000080:00200000:2.0:1789760782.814972:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72441856, bytes: 4096 00000080:00200000:2.0:1789760782.814981:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.814989:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.814992:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72441856, 72445952) 0 0 00000080:00200000:2.0:1789760782.814996:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.815013:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17686, 17686] 00020000:00200000:2.0:1789760782.815021:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72441856 chunk: [72441856, 72445952) 72445952, 4096 00020000:00200000:2.0:1789760782.815026:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.815045:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17686, 17686] 00000008:00010000:2.0:1789760782.815069:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.815074:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.815088:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72441856, 72445952) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.815101:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72441856=0x4516000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.815155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.815157:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72445952 00000020:00010000:2.0:1789760782.815165:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.815168:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.815171:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.815174:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.815177:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.815184:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.815189:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.815203:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.815314:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72441856<>72445952 00000100:00100000:2.0:1789760782.815361:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565494272:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.824601:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984994 gen 3 00000008:00010000:2.0:1789760782.824678:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760782.824687:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.824698:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.824776:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760782.824806:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565494272:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.825731:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17686, 17686] 00000080:00200000:0.0:1789760782.825753:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.825759:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.825777:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72445952 i_blocks 141520 00000020:00200000:0.0:1789760782.825781:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.825792:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.825826:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72445952, count: 4096 00000080:00200000:0.0:1789760782.825835:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72445952, bytes: 4096 00000080:00200000:0.0:1789760782.825847:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.825856:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.825859:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72445952, 72450048) 0 0 00000080:00200000:0.0:1789760782.825863:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.825881:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17687, 17687] 00020000:00200000:0.0:1789760782.825889:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72445952 chunk: [72445952, 72450048) 72450048, 4096 00020000:00200000:0.0:1789760782.825893:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.825914:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17687, 17687] 00000008:00010000:0.0:1789760782.825933:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.825939:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.825950:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72445952, 72450048) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.826036:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72445952=0x4517000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.826103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.826106:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72450048 00000020:00010000:0.0:1789760782.826115:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.826118:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.826121:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.826124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.826127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.826134:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.826141:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.826169:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.826299:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72445952<>72450048 00000100:00100000:1.0:1789760782.826378:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760782.828362:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565494400:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.844499:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984995 gen 3 00000008:00010000:0.0:1789760782.844563:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.844572:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.844587:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.844612:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.844665:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565494400:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.844689:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17687, 17687] 00000080:00200000:2.0:1789760782.844709:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.844715:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.844732:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72450048 i_blocks 141528 00000020:00200000:2.0:1789760782.844737:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.844750:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.844785:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72450048, count: 4096 00000080:00200000:2.0:1789760782.844798:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72450048, bytes: 4096 00000080:00200000:2.0:1789760782.844809:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.844818:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.844821:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72450048, 72454144) 0 0 00000080:00200000:2.0:1789760782.844826:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.844847:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17688, 17688] 00020000:00200000:2.0:1789760782.844857:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72450048 chunk: [72450048, 72454144) 72454144, 4096 00020000:00200000:2.0:1789760782.844862:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.844883:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17688, 17688] 00000008:00010000:2.0:1789760782.844901:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.844908:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.844920:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72450048, 72454144) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.844935:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72450048=0x4518000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.844987:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.844989:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72454144 00000020:00010000:2.0:1789760782.844996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.844998:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.845001:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.845004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.845006:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.845012:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.845017:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.845030:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.845134:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72450048<>72454144 00000100:00100000:2.0:1789760782.845172:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760782.845185:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565494528:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.860984:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984996 gen 3 00000008:00010000:2.0:1789760782.861033:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760782.861042:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.861054:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.861078:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760782.861109:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565494528:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.861385:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17688, 17688] 00000080:00200000:0.0:1789760782.861400:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.861406:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.861421:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72454144 i_blocks 141536 00000020:00200000:0.0:1789760782.861424:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.861435:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.861468:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72454144, count: 4096 00000080:00200000:0.0:1789760782.861483:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72454144, bytes: 4096 00000080:00200000:0.0:1789760782.861494:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.861504:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.861506:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72454144, 72458240) 0 0 00000080:00200000:0.0:1789760782.861511:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.861528:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17689, 17689] 00020000:00200000:0.0:1789760782.861538:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72454144 chunk: [72454144, 72458240) 72458240, 4096 00020000:00200000:0.0:1789760782.861542:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.861560:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17689, 17689] 00000008:00010000:0.0:1789760782.861577:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.861867:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.861880:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72454144, 72458240) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.861894:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72454144=0x4519000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.861956:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.861959:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72458240 00000020:00010000:0.0:1789760782.861968:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.861972:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.861975:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.861978:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.861981:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.861990:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.861997:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.862017:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.862144:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72454144<>72458240 00000100:00100000:0.0:1789760782.862206:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565494656:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760782.875070:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760782.880903:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984997 gen 3 00000008:00010000:0.0:1789760782.880954:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.880962:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.880973:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.880999:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.881028:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565494656:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.881277:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17689, 17689] 00000080:00200000:2.0:1789760782.881293:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.881310:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.881328:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72458240 i_blocks 141544 00000020:00200000:2.0:1789760782.881332:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.881344:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.881380:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72458240, count: 4096 00000080:00200000:2.0:1789760782.881388:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72458240, bytes: 4096 00000080:00200000:2.0:1789760782.881399:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.881408:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.881410:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72458240, 72462336) 0 0 00000080:00200000:2.0:1789760782.881414:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.881431:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17690, 17690] 00020000:00200000:2.0:1789760782.881440:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72458240 chunk: [72458240, 72462336) 72462336, 4096 00020000:00200000:2.0:1789760782.881445:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.881485:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17690, 17690] 00000008:00010000:2.0:1789760782.881505:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.881511:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.881523:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72458240, 72462336) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.881537:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72458240=0x451a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.881608:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.881611:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72462336 00000020:00010000:2.0:1789760782.881619:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.881622:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.881625:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.881628:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.881630:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.881638:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.881680:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.881697:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.881817:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72458240<>72462336 00000100:00100000:2.0:1789760782.881873:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565494784:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.900132:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984998 gen 3 00000008:00010000:2.0:1789760782.900182:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760782.900190:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.900201:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.900317:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760782.900360:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565494784:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.901295:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17690, 17690] 00000080:00200000:0.0:1789760782.901319:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.901327:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.901603:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72462336 i_blocks 141552 00000020:00200000:0.0:1789760782.901608:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.901621:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.901803:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72462336, count: 4096 00000080:00200000:0.0:1789760782.901813:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72462336, bytes: 4096 00000080:00200000:0.0:1789760782.901828:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.901838:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.901840:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72462336, 72466432) 0 0 00000080:00200000:0.0:1789760782.901845:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.901862:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17691, 17691] 00020000:00200000:0.0:1789760782.901871:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72462336 chunk: [72462336, 72466432) 72466432, 4096 00020000:00200000:0.0:1789760782.901875:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.901895:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17691, 17691] 00000008:00010000:0.0:1789760782.901913:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.901918:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.901930:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72462336, 72466432) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.901945:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72462336=0x451b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.902205:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.902209:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72466432 00000020:00010000:0.0:1789760782.902218:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.902221:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.902224:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.902228:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.902231:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.902240:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.902248:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.902278:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.903143:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72462336<>72466432 00000100:00100000:1.0:1789760782.903222:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565494912:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.917207:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294984999 gen 3 00000008:00010000:0.0:1789760782.917264:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.917274:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.917286:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.917314:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.917343:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565494912:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.918954:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17691, 17691] 00000080:00200000:2.0:1789760782.918976:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.918984:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.919003:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72466432 i_blocks 141560 00000020:00200000:2.0:1789760782.919009:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.919026:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.919063:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72466432, count: 4096 00000080:00200000:2.0:1789760782.919071:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72466432, bytes: 4096 00000080:00200000:2.0:1789760782.919085:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.919095:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.919097:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72466432, 72470528) 0 0 00000080:00200000:2.0:1789760782.919102:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.919119:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17692, 17692] 00020000:00200000:2.0:1789760782.919129:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72466432 chunk: [72466432, 72470528) 72470528, 4096 00020000:00200000:2.0:1789760782.919133:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.919157:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17692, 17692] 00000008:00010000:2.0:1789760782.919176:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.919182:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.919194:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72466432, 72470528) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.919208:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72466432=0x451c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.919279:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.919282:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72470528 00000020:00010000:2.0:1789760782.919291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.919294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.919297:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.919301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.919305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.919313:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.919319:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.919344:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.919488:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72466432<>72470528 00000100:00100000:2.0:1789760782.919541:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760782.919566:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565495040:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.931858:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985000 gen 3 00000008:00010000:2.0:1789760782.931921:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760782.931933:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.931945:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.931976:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760782.932009:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565495040:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.932464:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17692, 17692] 00000080:00200000:0.0:1789760782.932797:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.932805:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.932829:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72470528 i_blocks 141568 00000020:00200000:0.0:1789760782.932834:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.932846:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.932883:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72470528, count: 4096 00000080:00200000:0.0:1789760782.932890:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72470528, bytes: 4096 00000080:00200000:0.0:1789760782.932905:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.932915:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.932917:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72470528, 72474624) 0 0 00000080:00200000:0.0:1789760782.932921:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.932939:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17693, 17693] 00020000:00200000:0.0:1789760782.932947:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72470528 chunk: [72470528, 72474624) 72474624, 4096 00020000:00200000:0.0:1789760782.932951:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.932971:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17693, 17693] 00000008:00010000:0.0:1789760782.932989:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.932995:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.933008:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72470528, 72474624) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.933023:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72470528=0x451d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.933085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.933088:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72474624 00000020:00010000:0.0:1789760782.933095:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.933099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.933102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.933105:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.933108:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.933117:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.933124:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.933146:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.933275:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72470528<>72474624 00000100:00100000:0.0:1789760782.933331:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565495168:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.945042:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985001 gen 3 00000008:00010000:0.0:1789760782.945118:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.945126:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.945138:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.945164:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.945212:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565495168:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.946566:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17693, 17693] 00000080:00200000:2.0:1789760782.946589:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.946595:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.946614:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72474624 i_blocks 141576 00000020:00200000:2.0:1789760782.946619:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.946635:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.946704:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72474624, count: 4096 00000080:00200000:2.0:1789760782.946712:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72474624, bytes: 4096 00000080:00200000:2.0:1789760782.946728:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.946738:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.946740:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72474624, 72478720) 0 0 00000080:00200000:2.0:1789760782.946745:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.946763:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17694, 17694] 00020000:00200000:2.0:1789760782.946772:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72474624 chunk: [72474624, 72478720) 72478720, 4096 00020000:00200000:2.0:1789760782.946777:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.946798:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17694, 17694] 00000008:00010000:2.0:1789760782.946819:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.946834:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.946847:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72474624, 72478720) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.946862:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72474624=0x451e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.946928:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.946931:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72478720 00000020:00010000:2.0:1789760782.946940:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.946944:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.946947:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.946951:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.946953:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.946962:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.946969:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.946992:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.947130:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72474624<>72478720 00000100:00100000:2.0:1789760782.947198:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565495296:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.960915:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985002 gen 3 00000008:00010000:2.0:1789760782.960974:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760782.960984:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.960997:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.961040:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760782.961076:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565495296:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.963971:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17694, 17694] 00000080:00200000:0.0:1789760782.963998:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.964015:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.964037:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72478720 i_blocks 141584 00000020:00200000:0.0:1789760782.964043:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.964061:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.964097:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72478720, count: 4096 00000080:00200000:0.0:1789760782.964107:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72478720, bytes: 4096 00000080:00200000:0.0:1789760782.964123:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.964132:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.964135:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72478720, 72482816) 0 0 00000080:00200000:0.0:1789760782.964140:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.964158:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17695, 17695] 00020000:00200000:0.0:1789760782.964168:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72478720 chunk: [72478720, 72482816) 72482816, 4096 00020000:00200000:0.0:1789760782.964174:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.964196:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17695, 17695] 00000008:00010000:0.0:1789760782.964213:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.964219:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.964232:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72478720, 72482816) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.964245:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72478720=0x451f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.964319:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.964322:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72482816 00000020:00010000:0.0:1789760782.964330:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.964334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.964337:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.964340:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.964343:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.964353:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.964360:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.964386:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.964545:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72478720<>72482816 00000100:00100000:0.0:1789760782.964598:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760782.964626:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565495424:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.973720:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985003 gen 3 00000008:00010000:0.0:1789760782.973768:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.973776:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.973788:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.973817:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.973849:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565495424:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.973972:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17695, 17695] 00000080:00200000:2.0:1789760782.973988:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.973994:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.974008:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72482816 i_blocks 141592 00000020:00200000:2.0:1789760782.974012:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.974023:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.974058:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72482816, count: 4096 00000080:00200000:2.0:1789760782.974065:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72482816, bytes: 4096 00000080:00200000:2.0:1789760782.974074:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.974083:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.974086:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72482816, 72486912) 0 0 00000080:00200000:2.0:1789760782.974090:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.974108:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17696, 17696] 00020000:00200000:2.0:1789760782.974117:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72482816 chunk: [72482816, 72486912) 72486912, 4096 00020000:00200000:2.0:1789760782.974122:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.974140:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17696, 17696] 00000008:00010000:2.0:1789760782.974157:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.974164:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.974176:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72482816, 72486912) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.974190:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72482816=0x4520000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.974247:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.974250:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72486912 00000020:00010000:2.0:1789760782.974258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.974276:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.974280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.974283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.974286:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.974293:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.974299:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760782.974316:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760782.974437:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72482816<>72486912 00000100:00100000:2.0:1789760782.974492:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760782.974518:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565495552:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760782.988154:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985004 gen 3 00000008:00010000:2.0:1789760782.988207:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760782.988217:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760782.988228:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760782.988258:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760782.988308:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565495552:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760782.988737:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17696, 17696] 00000080:00200000:0.0:1789760782.988761:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760782.988767:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760782.988783:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72486912 i_blocks 141600 00000020:00200000:0.0:1789760782.988787:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760782.988797:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760782.988902:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72486912, count: 4096 00000080:00200000:0.0:1789760782.988912:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72486912, bytes: 4096 00000080:00200000:0.0:1789760782.988924:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760782.988933:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760782.988935:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72486912, 72491008) 0 0 00000080:00200000:0.0:1789760782.988940:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760782.988956:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17697, 17697] 00020000:00200000:0.0:1789760782.988965:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72486912 chunk: [72486912, 72491008) 72491008, 4096 00020000:00200000:0.0:1789760782.988970:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760782.988989:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17697, 17697] 00000008:00010000:0.0:1789760782.989007:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760782.989014:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760782.989024:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72486912, 72491008) append 0 append lockpos 0 00000080:00200000:0.0:1789760782.989038:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72486912=0x4521000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760782.989096:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760782.989098:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72491008 00000020:00010000:0.0:1789760782.989107:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760782.989110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.989113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760782.989117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760782.989120:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760782.989127:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760782.989134:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760782.989153:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760782.989273:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72486912<>72491008 00000100:00100000:1.0:1789760782.989337:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760782.989465:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565495680:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760782.999129:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985005 gen 3 00000008:00010000:0.0:1789760782.999180:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760782.999188:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760782.999199:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760782.999225:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760782.999254:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565495680:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760782.999376:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17697, 17697] 00000080:00200000:2.0:1789760782.999394:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760782.999400:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760782.999426:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72491008 i_blocks 141608 00000020:00200000:2.0:1789760782.999430:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760782.999443:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760782.999741:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72491008, count: 4096 00000080:00200000:2.0:1789760782.999751:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72491008, bytes: 4096 00000080:00200000:2.0:1789760782.999764:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760782.999773:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760782.999776:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72491008, 72495104) 0 0 00000080:00200000:2.0:1789760782.999780:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760782.999799:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17698, 17698] 00020000:00200000:2.0:1789760782.999809:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72491008 chunk: [72491008, 72495104) 72495104, 4096 00020000:00200000:2.0:1789760782.999813:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760782.999844:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17698, 17698] 00000008:00010000:2.0:1789760782.999863:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760782.999869:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760782.999881:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72491008, 72495104) append 0 append lockpos 0 00000080:00200000:2.0:1789760782.999895:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72491008=0x4522000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760782.999950:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760782.999953:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72495104 00000020:00010000:2.0:1789760782.999961:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760782.999964:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.999967:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760782.999971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760782.999974:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760782.999981:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760782.999988:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.000007:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.000134:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72491008<>72495104 00000100:00100000:2.0:1789760783.000195:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565495808:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.010365:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985006 gen 3 00000008:00010000:2.0:1789760783.010415:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760783.010424:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.010435:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.010464:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760783.010517:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565495808:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.010731:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17698, 17698] 00000080:00200000:0.0:1789760783.010749:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.010755:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.010772:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72495104 i_blocks 141616 00000020:00200000:0.0:1789760783.010776:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.010788:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.010842:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72495104, count: 4096 00000080:00200000:0.0:1789760783.010850:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72495104, bytes: 4096 00000080:00200000:0.0:1789760783.010861:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.010871:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.010874:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72495104, 72499200) 0 0 00000080:00200000:0.0:1789760783.010878:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.010896:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17699, 17699] 00020000:00200000:0.0:1789760783.010906:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72495104 chunk: [72495104, 72499200) 72499200, 4096 00020000:00200000:0.0:1789760783.010911:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.010932:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17699, 17699] 00000008:00010000:0.0:1789760783.010950:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.010956:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.010968:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72495104, 72499200) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.010984:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72495104=0x4523000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.011045:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.011048:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72499200 00000020:00010000:0.0:1789760783.011057:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.011060:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.011063:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.011067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.011070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.011078:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.011085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760783.011101:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760783.011228:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72495104<>72499200 00000100:00100000:0.0:1789760783.011284:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760783.011302:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565495936:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.019736:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985007 gen 3 00000008:00010000:0.0:1789760783.019786:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760783.019794:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.019805:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.019885:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760783.019915:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565495936:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.020137:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17699, 17699] 00000080:00200000:2.0:1789760783.020154:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.020160:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.020175:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72499200 i_blocks 141624 00000020:00200000:2.0:1789760783.020179:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.020191:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.020226:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72499200, count: 4096 00000080:00200000:2.0:1789760783.020233:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72499200, bytes: 4096 00000080:00200000:2.0:1789760783.020245:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.020254:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.020257:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72499200, 72503296) 0 0 00000080:00200000:2.0:1789760783.020261:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.020279:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17700, 17700] 00020000:00200000:2.0:1789760783.020288:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72499200 chunk: [72499200, 72503296) 72503296, 4096 00020000:00200000:2.0:1789760783.020293:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.020312:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17700, 17700] 00000008:00010000:2.0:1789760783.020330:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.020336:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.020349:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72499200, 72503296) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.020362:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72499200=0x4524000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.020745:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.020749:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72503296 00000020:00010000:2.0:1789760783.020759:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.020763:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.020766:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.020770:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.020773:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.020783:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.020790:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.020814:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.021013:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72499200<>72503296 00000100:00100000:3.0:1789760783.021100:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565496064:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.031252:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985008 gen 3 00000008:00010000:3.0:1789760783.031308:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760783.031318:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.031329:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.031352:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760783.031395:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565496064:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.031719:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17700, 17700] 00000080:00200000:2.0:1789760783.031736:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.031743:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.031761:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72503296 i_blocks 141632 00000020:00200000:2.0:1789760783.031766:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.031776:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.031811:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72503296, count: 4096 00000080:00200000:2.0:1789760783.031818:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72503296, bytes: 4096 00000080:00200000:2.0:1789760783.031902:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.031911:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.031914:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72503296, 72507392) 0 0 00000080:00200000:2.0:1789760783.031919:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.031937:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17701, 17701] 00020000:00200000:2.0:1789760783.031947:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72503296 chunk: [72503296, 72507392) 72507392, 4096 00020000:00200000:2.0:1789760783.031952:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.031974:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17701, 17701] 00000008:00010000:2.0:1789760783.031991:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.031997:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.032009:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72503296, 72507392) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.032023:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72503296=0x4525000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.032082:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.032085:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72507392 00000020:00010000:2.0:1789760783.032093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.032096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.032099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.032102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.032104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.032113:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.032119:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.032136:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760783.032264:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 72503296<>72507392 00000100:00100000:3.0:1789760783.032335:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760783.033252:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565496192:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.044137:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985009 gen 3 00000008:00010000:3.0:1789760783.044181:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760783.044189:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.044199:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.044220:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760783.044246:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565496192:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.044351:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17701, 17701] 00000080:00200000:2.0:1789760783.044368:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.044374:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.044388:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72507392 i_blocks 141640 00000020:00200000:2.0:1789760783.044391:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.044401:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.044432:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72507392, count: 4096 00000080:00200000:2.0:1789760783.044438:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72507392, bytes: 4096 00000080:00200000:2.0:1789760783.044447:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.044456:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.044459:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72507392, 72511488) 0 0 00000080:00200000:2.0:1789760783.044463:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.044492:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17702, 17702] 00020000:00200000:2.0:1789760783.044509:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72507392 chunk: [72507392, 72511488) 72511488, 4096 00020000:00200000:2.0:1789760783.044514:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.044532:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17702, 17702] 00000008:00010000:2.0:1789760783.044548:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.044554:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.044564:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72507392, 72511488) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.044576:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72507392=0x4526000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.044630:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.044632:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72511488 00000020:00010000:2.0:1789760783.044640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.044681:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.044685:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.044688:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.044691:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.044700:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.044707:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.044725:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760783.044845:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 72507392<>72511488 00000100:00100000:2.0:1789760783.044884:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760783.044899:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565496320:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.062122:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985010 gen 3 00000008:00010000:2.0:1789760783.062239:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760783.062248:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.062259:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.062289:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760783.062320:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565496320:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.062435:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17702, 17702] 00000080:00200000:0.0:1789760783.062452:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.062457:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.062470:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72511488 i_blocks 141648 00000020:00200000:0.0:1789760783.062482:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.062493:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.062529:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72511488, count: 4096 00000080:00200000:0.0:1789760783.062537:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72511488, bytes: 4096 00000080:00200000:0.0:1789760783.062550:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.062559:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.062562:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72511488, 72515584) 0 0 00000080:00200000:0.0:1789760783.062566:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.062583:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17703, 17703] 00020000:00200000:0.0:1789760783.062596:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72511488 chunk: [72511488, 72515584) 72515584, 4096 00020000:00200000:0.0:1789760783.062600:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.062620:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17703, 17703] 00000008:00010000:0.0:1789760783.062639:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.062680:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.062693:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72511488, 72515584) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.063018:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72511488=0x4527000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.063084:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.063088:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72515584 00000020:00010000:0.0:1789760783.063096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.063100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.063103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.063106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.063109:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.063118:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.063124:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760783.063200:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760783.063331:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72511488<>72515584 00000100:00100000:1.0:1789760783.063486:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565496448:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.074331:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985011 gen 3 00000008:00010000:0.0:1789760783.074515:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760783.074524:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.074537:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.074616:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760783.074687:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565496448:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.075020:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17703, 17703] 00000080:00200000:2.0:1789760783.075038:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.075044:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.075059:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72515584 i_blocks 141656 00000020:00200000:2.0:1789760783.075063:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.075075:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.075111:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72515584, count: 4096 00000080:00200000:2.0:1789760783.075119:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72515584, bytes: 4096 00000080:00200000:2.0:1789760783.075129:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.075140:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.075142:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72515584, 72519680) 0 0 00000080:00200000:2.0:1789760783.075147:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.075165:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17704, 17704] 00020000:00200000:2.0:1789760783.075173:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72515584 chunk: [72515584, 72519680) 72519680, 4096 00020000:00200000:2.0:1789760783.075178:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.075198:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17704, 17704] 00000008:00010000:2.0:1789760783.075217:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.075223:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.075234:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72515584, 72519680) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.075250:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72515584=0x4528000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.075308:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.075312:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72519680 00000020:00010000:2.0:1789760783.075321:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.075324:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.075327:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.075330:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.075333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.075340:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.075346:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.075377:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760783.075610:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 72515584<>72519680 00000100:00100000:3.0:1789760783.075684:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760783.075887:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565496576:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.090011:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985012 gen 3 00000008:00010000:3.0:1789760783.090072:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760783.090081:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.090095:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.090178:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760783.090211:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565496576:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.090948:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17704, 17704] 00000080:00200000:2.0:1789760783.090966:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.090973:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.090988:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72519680 i_blocks 141664 00000020:00200000:2.0:1789760783.090993:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.091004:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.091053:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72519680, count: 4096 00000080:00200000:2.0:1789760783.091061:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72519680, bytes: 4096 00000080:00200000:2.0:1789760783.091072:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.091081:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.091084:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72519680, 72523776) 0 0 00000080:00200000:2.0:1789760783.091089:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.091108:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17705, 17705] 00020000:00200000:2.0:1789760783.091117:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72519680 chunk: [72519680, 72523776) 72523776, 4096 00020000:00200000:2.0:1789760783.091122:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.091143:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17705, 17705] 00000008:00010000:2.0:1789760783.091160:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.091166:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.091177:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72519680, 72523776) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.091192:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72519680=0x4529000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.091253:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.091256:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72523776 00000020:00010000:2.0:1789760783.091265:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.091268:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.091271:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.091274:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.091277:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.091284:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.091291:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.091315:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760783.091725:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 72519680<>72523776 00000100:00100000:2.0:1789760783.091772:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760783.091793:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565496704:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.102777:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985013 gen 3 00000008:00010000:2.0:1789760783.102823:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760783.102832:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.102842:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.102875:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760783.102903:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565496704:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.103154:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17705, 17705] 00000080:00200000:0.0:1789760783.103172:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.103178:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.103192:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72523776 i_blocks 141672 00000020:00200000:0.0:1789760783.103196:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.103205:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.103241:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72523776, count: 4096 00000080:00200000:0.0:1789760783.103249:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72523776, bytes: 4096 00000080:00200000:0.0:1789760783.103259:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.103267:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.103270:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72523776, 72527872) 0 0 00000080:00200000:0.0:1789760783.103274:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.103290:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17706, 17706] 00020000:00200000:0.0:1789760783.103299:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72523776 chunk: [72523776, 72527872) 72527872, 4096 00020000:00200000:0.0:1789760783.103305:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.103327:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17706, 17706] 00000008:00010000:0.0:1789760783.103346:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.103352:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.103364:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72523776, 72527872) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.103378:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72523776=0x452a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.103634:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.103638:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72527872 00000020:00010000:0.0:1789760783.104019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.104024:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.104027:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.104031:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.104034:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.104044:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.104052:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760783.104076:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760783.104210:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72523776<>72527872 00000100:00100000:0.0:1789760783.104289:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760783.104312:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565496832:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.114264:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985014 gen 3 00000008:00010000:0.0:1789760783.114316:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760783.114324:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.114335:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.114363:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760783.114393:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565496832:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.114807:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17706, 17706] 00000080:00200000:1.0:1789760783.114834:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.114841:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.114865:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72527872 i_blocks 141680 00000020:00200000:1.0:1789760783.114870:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.114885:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.114921:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72527872, count: 4096 00000080:00200000:1.0:1789760783.114928:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72527872, bytes: 4096 00000080:00200000:1.0:1789760783.114939:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.114948:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.114951:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72527872, 72531968) 0 0 00000080:00200000:1.0:1789760783.114956:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.114972:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17707, 17707] 00020000:00200000:1.0:1789760783.114980:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72527872 chunk: [72527872, 72531968) 72531968, 4096 00020000:00200000:1.0:1789760783.114984:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.115004:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17707, 17707] 00000008:00010000:1.0:1789760783.115021:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.115028:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.115039:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72527872, 72531968) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.115052:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72527872=0x452b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.115114:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.115117:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72531968 00000020:00010000:1.0:1789760783.115126:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.115130:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.115133:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.115136:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.115139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.115147:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.115155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.115171:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760783.115300:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 72527872<>72531968 00000100:00100000:1.0:1789760783.115345:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760783.115363:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565496960:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.124398:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985015 gen 3 00000008:00010000:0.0:1789760783.124458:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760783.124467:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.124936:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.125010:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760783.125041:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565496960:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.125134:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17707, 17707] 00000080:00200000:1.0:1789760783.125151:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.125156:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.125172:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72531968 i_blocks 141688 00000020:00200000:1.0:1789760783.125176:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.125186:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.125218:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72531968, count: 4096 00000080:00200000:1.0:1789760783.125225:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72531968, bytes: 4096 00000080:00200000:1.0:1789760783.125236:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.125245:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.125247:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72531968, 72536064) 0 0 00000080:00200000:1.0:1789760783.125251:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.125267:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17708, 17708] 00020000:00200000:1.0:1789760783.125276:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72531968 chunk: [72531968, 72536064) 72536064, 4096 00020000:00200000:1.0:1789760783.125280:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.125300:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17708, 17708] 00000008:00010000:1.0:1789760783.125317:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.125323:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.125335:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72531968, 72536064) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.125348:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72531968=0x452c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.125405:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.125727:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72536064 00000020:00010000:1.0:1789760783.125740:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.125744:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.125748:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.125752:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.125756:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.125764:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.125772:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.125792:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760783.126031:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 72531968<>72536064 00000100:00100000:0.0:1789760783.126095:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497088:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.138734:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985016 gen 3 00000008:00010000:0.0:1789760783.138785:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760783.138793:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.138805:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.138826:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760783.138856:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497088:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.139172:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17708, 17708] 00000080:00200000:1.0:1789760783.139189:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.139194:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.139209:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72536064 i_blocks 141696 00000020:00200000:1.0:1789760783.139214:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.139223:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.139258:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72536064, count: 4096 00000080:00200000:1.0:1789760783.139265:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72536064, bytes: 4096 00000080:00200000:1.0:1789760783.139277:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.139286:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.139289:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72536064, 72540160) 0 0 00000080:00200000:1.0:1789760783.139293:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.139311:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17709, 17709] 00020000:00200000:1.0:1789760783.139323:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72536064 chunk: [72536064, 72540160) 72540160, 4096 00020000:00200000:1.0:1789760783.139329:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.139349:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17709, 17709] 00000008:00010000:1.0:1789760783.139366:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.139372:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.139385:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72536064, 72540160) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.139399:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72536064=0x452d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.139472:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.139484:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72540160 00000020:00010000:1.0:1789760783.139492:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.139496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.139499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.139503:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.139507:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.139514:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.139521:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.139538:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760783.139713:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 72536064<>72540160 00000100:00100000:0.0:1789760783.139866:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760783.139886:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497216:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.150287:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985017 gen 3 00000008:00010000:0.0:1789760783.150340:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760783.150349:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.150362:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.150383:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760783.150413:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497216:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.150587:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17709, 17709] 00000080:00200000:1.0:1789760783.150604:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.150609:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.150623:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72540160 i_blocks 141704 00000020:00200000:1.0:1789760783.150627:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.150637:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.150704:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72540160, count: 4096 00000080:00200000:1.0:1789760783.150712:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72540160, bytes: 4096 00000080:00200000:1.0:1789760783.150722:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.150733:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.150736:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72540160, 72544256) 0 0 00000080:00200000:1.0:1789760783.150740:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.150758:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17710, 17710] 00020000:00200000:1.0:1789760783.150767:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72540160 chunk: [72540160, 72544256) 72544256, 4096 00020000:00200000:1.0:1789760783.150771:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.150791:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17710, 17710] 00000008:00010000:1.0:1789760783.150810:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.150816:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.150840:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72540160, 72544256) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.150855:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72540160=0x452e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.150916:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.150918:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72544256 00000020:00010000:1.0:1789760783.150927:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.150930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.150933:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.150937:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.150941:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.150949:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.150956:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.150973:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760783.151226:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 72540160<>72544256 00000100:00100000:0.0:1789760783.151525:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497344:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.162950:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985018 gen 3 00000008:00010000:0.0:1789760783.163002:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760783.163179:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.163192:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.163215:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760783.163249:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497344:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.163900:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17710, 17710] 00000080:00200000:1.0:1789760783.163920:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.163926:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.163944:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72544256 i_blocks 141712 00000020:00200000:1.0:1789760783.163948:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.163958:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.163991:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72544256, count: 4096 00000080:00200000:1.0:1789760783.163999:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72544256, bytes: 4096 00000080:00200000:1.0:1789760783.164695:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.164708:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.164710:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72544256, 72548352) 0 0 00000080:00200000:1.0:1789760783.164716:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.164734:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17711, 17711] 00020000:00200000:1.0:1789760783.164744:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72544256 chunk: [72544256, 72548352) 72548352, 4096 00020000:00200000:1.0:1789760783.164748:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.164769:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17711, 17711] 00000008:00010000:1.0:1789760783.164789:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.164795:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.164806:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72544256, 72548352) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.165140:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72544256=0x452f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.165216:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.165220:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72548352 00000020:00010000:1.0:1789760783.165229:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.165233:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.165236:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.165240:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.165244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.165254:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.165262:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.165288:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760783.165430:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 72544256<>72548352 00000100:00100000:0.0:1789760783.166276:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760783.166306:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497472:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.178201:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985019 gen 3 00000008:00010000:0.0:1789760783.178256:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760783.178265:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.178276:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.178390:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760783.178431:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497472:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.183066:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17711, 17711] 00000080:00200000:1.0:1789760783.183094:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.183100:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.183120:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72548352 i_blocks 141720 00000020:00200000:1.0:1789760783.183127:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.183144:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.183190:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72548352, count: 4096 00000080:00200000:1.0:1789760783.183199:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72548352, bytes: 4096 00000080:00200000:1.0:1789760783.183216:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.183226:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.183228:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72548352, 72552448) 0 0 00000080:00200000:1.0:1789760783.183234:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.183251:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17712, 17712] 00020000:00200000:1.0:1789760783.183260:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72548352 chunk: [72548352, 72552448) 72552448, 4096 00020000:00200000:1.0:1789760783.183265:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.183288:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17712, 17712] 00000008:00010000:1.0:1789760783.183306:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.183313:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.183325:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72548352, 72552448) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.183340:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72548352=0x4530000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.183414:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.183417:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72552448 00000020:00010000:1.0:1789760783.183426:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.183430:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.183433:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.183437:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.183441:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.183450:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.183457:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.183987:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760783.184126:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 72548352<>72552448 00000100:00100000:0.0:1789760783.184303:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497600:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.205051:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985020 gen 3 00000008:00010000:0.0:1789760783.205101:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760783.205110:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.205121:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.205243:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760783.205276:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497600:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.208737:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17712, 17712] 00000080:00200000:1.0:1789760783.208766:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.208773:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.208835:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72552448 i_blocks 141728 00000020:00200000:1.0:1789760783.208840:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.208853:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.208887:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72552448, count: 4096 00000080:00200000:1.0:1789760783.208896:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72552448, bytes: 4096 00000080:00200000:1.0:1789760783.208910:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.208921:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.208923:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72552448, 72556544) 0 0 00000080:00200000:1.0:1789760783.208927:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.208945:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17713, 17713] 00020000:00200000:1.0:1789760783.208954:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72552448 chunk: [72552448, 72556544) 72556544, 4096 00020000:00200000:1.0:1789760783.208959:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.208980:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17713, 17713] 00000008:00010000:1.0:1789760783.208998:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.209003:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.209014:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72552448, 72556544) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.209028:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72552448=0x4531000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.209086:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.209089:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72556544 00000020:00010000:1.0:1789760783.209097:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.209100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.209103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.209106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.209110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.209118:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.209125:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.209140:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760783.209272:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 72552448<>72556544 00000100:00100000:0.0:1789760783.209717:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760783.209747:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497728:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.226859:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985021 gen 3 00000008:00010000:0.0:1789760783.226911:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760783.226919:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.226931:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.226942:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760783.226972:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497728:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.227120:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17713, 17713] 00000080:00200000:1.0:1789760783.227137:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.227143:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.227158:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72556544 i_blocks 141736 00000020:00200000:1.0:1789760783.227162:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.227171:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.227206:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72556544, count: 4096 00000080:00200000:1.0:1789760783.227214:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72556544, bytes: 4096 00000080:00200000:1.0:1789760783.227453:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.227465:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.227467:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72556544, 72560640) 0 0 00000080:00200000:1.0:1789760783.227480:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.227500:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17714, 17714] 00020000:00200000:1.0:1789760783.227510:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72556544 chunk: [72556544, 72560640) 72560640, 4096 00020000:00200000:1.0:1789760783.227514:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.227536:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17714, 17714] 00000008:00010000:1.0:1789760783.227556:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.227562:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.227574:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72556544, 72560640) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.227588:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72556544=0x4532000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.228417:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.228425:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72560640 00000020:00010000:1.0:1789760783.228438:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.228442:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.228446:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.228449:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.228453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.228464:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.228483:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.228514:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760783.228820:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 72556544<>72560640 00000100:00100000:0.0:1789760783.229351:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497856:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.246619:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985022 gen 3 00000008:00010000:0.0:1789760783.246738:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760783.246748:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.246760:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.246775:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760783.246803:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497856:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.247103:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17714, 17714] 00000080:00200000:1.0:1789760783.247121:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.247128:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.247146:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72560640 i_blocks 141744 00000020:00200000:1.0:1789760783.247150:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.247160:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.247198:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72560640, count: 4096 00000080:00200000:1.0:1789760783.247206:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72560640, bytes: 4096 00000080:00200000:1.0:1789760783.247222:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.247232:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.247234:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72560640, 72564736) 0 0 00000080:00200000:1.0:1789760783.247239:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.247258:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17715, 17715] 00020000:00200000:1.0:1789760783.247267:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72560640 chunk: [72560640, 72564736) 72564736, 4096 00020000:00200000:1.0:1789760783.247272:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.247293:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17715, 17715] 00000008:00010000:1.0:1789760783.247313:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.247319:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.247331:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72560640, 72564736) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.247345:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72560640=0x4533000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.247407:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.247411:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72564736 00000020:00010000:1.0:1789760783.247420:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.247424:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.247426:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.247430:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.247434:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.247442:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.247449:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.247466:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760783.247606:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 72560640<>72564736 00000100:00100000:0.0:1789760783.247900:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760783.247932:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497984:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.265335:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985023 gen 3 00000008:00010000:0.0:1789760783.265389:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760783.265397:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.265409:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.265467:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760783.265511:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565497984:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.265792:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17715, 17715] 00000080:00200000:1.0:1789760783.265809:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.265815:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.265893:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72564736 i_blocks 141752 00000020:00200000:1.0:1789760783.265898:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.265908:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.265941:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72564736, count: 4096 00000080:00200000:1.0:1789760783.265948:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72564736, bytes: 4096 00000080:00200000:1.0:1789760783.265960:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.265969:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.265971:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72564736, 72568832) 0 0 00000080:00200000:1.0:1789760783.265976:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.265992:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17716, 17716] 00020000:00200000:1.0:1789760783.266000:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72564736 chunk: [72564736, 72568832) 72568832, 4096 00020000:00200000:1.0:1789760783.266005:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.266024:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17716, 17716] 00000008:00010000:1.0:1789760783.266042:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.266114:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.266128:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72564736, 72568832) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.266142:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72564736=0x4534000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.266198:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.266200:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72568832 00000020:00010000:1.0:1789760783.266209:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.266213:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.266216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.266219:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.266223:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.266230:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.266237:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.266255:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760783.266373:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 72564736<>72568832 00000100:00100000:0.0:1789760783.266443:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565498112:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.282192:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985024 gen 3 00000008:00010000:0.0:1789760783.282244:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760783.282253:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.282263:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.282284:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760783.282315:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565498112:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.282426:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17716, 17716] 00000080:00200000:1.0:1789760783.282443:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.282449:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.282464:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72568832 i_blocks 141760 00000020:00200000:1.0:1789760783.282468:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.282536:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.282574:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72568832, count: 4096 00000080:00200000:1.0:1789760783.282583:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72568832, bytes: 4096 00000080:00200000:1.0:1789760783.282595:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.282603:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.282606:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72568832, 72572928) 0 0 00000080:00200000:1.0:1789760783.282610:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.282629:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17717, 17717] 00020000:00200000:1.0:1789760783.282638:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72568832 chunk: [72568832, 72572928) 72572928, 4096 00020000:00200000:1.0:1789760783.282765:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.282787:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17717, 17717] 00000008:00010000:1.0:1789760783.282804:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.282810:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.282956:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72568832, 72572928) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.282971:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72568832=0x4535000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.283157:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.283161:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72572928 00000020:00010000:1.0:1789760783.283169:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.283173:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.283176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.283180:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.283183:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.283191:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.283198:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.283217:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760783.283342:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 72568832<>72572928 00000100:00100000:0.0:1789760783.283440:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760783.283460:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565498240:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.297166:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985025 gen 3 00000008:00010000:0.0:1789760783.297219:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760783.297228:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.297239:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.297261:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760783.297292:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565498240:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.297674:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17717, 17717] 00000080:00200000:1.0:1789760783.297692:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.297698:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.297713:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72572928 i_blocks 141768 00000020:00200000:1.0:1789760783.297716:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.297725:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.297760:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72572928, count: 4096 00000080:00200000:1.0:1789760783.297768:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72572928, bytes: 4096 00000080:00200000:1.0:1789760783.297779:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.297788:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.297790:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72572928, 72577024) 0 0 00000080:00200000:1.0:1789760783.297795:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.297812:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17718, 17718] 00020000:00200000:1.0:1789760783.297834:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72572928 chunk: [72572928, 72577024) 72577024, 4096 00020000:00200000:1.0:1789760783.297839:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.297861:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17718, 17718] 00000008:00010000:1.0:1789760783.297881:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.297886:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.297898:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72572928, 72577024) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.297913:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72572928=0x4536000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.297972:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.297975:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72577024 00000020:00010000:1.0:1789760783.297983:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.300205:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.300214:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.300219:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.300224:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.300242:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.300255:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.300294:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760783.300441:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 72572928<>72577024 00000100:00100000:0.0:1789760783.304016:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565498368:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.317022:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985026 gen 3 00000008:00010000:0.0:1789760783.317082:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760783.317091:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.317102:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.317124:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760783.317156:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565498368:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.317160:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17718, 17718] 00000080:00200000:1.0:1789760783.317181:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.317189:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.317207:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72577024 i_blocks 141776 00000020:00200000:1.0:1789760783.317211:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.317222:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.317258:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72577024, count: 4096 00000080:00200000:1.0:1789760783.317266:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72577024, bytes: 4096 00000080:00200000:1.0:1789760783.317277:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.317286:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.317288:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72577024, 72581120) 0 0 00000080:00200000:1.0:1789760783.317293:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.317311:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17719, 17719] 00020000:00200000:1.0:1789760783.317319:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72577024 chunk: [72577024, 72581120) 72581120, 4096 00020000:00200000:1.0:1789760783.317324:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.317342:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17719, 17719] 00000008:00010000:1.0:1789760783.317361:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.317367:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.317378:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72577024, 72581120) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.317392:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72577024=0x4537000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.317589:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.317593:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72581120 00000020:00010000:1.0:1789760783.317601:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.317605:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.317608:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.317611:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.317615:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.317622:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.317629:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.319611:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760783.320007:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 72577024<>72581120 00000100:00100000:0.0:1789760783.320520:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760783.320547:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565498496:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.341978:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985027 gen 3 00000008:00010000:0.0:1789760783.342038:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760783.342046:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.342058:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.342162:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760783.342192:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565498496:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.344837:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17719, 17719] 00000080:00200000:2.0:1789760783.344864:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.344871:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.344888:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72581120 i_blocks 141784 00000020:00200000:2.0:1789760783.344893:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.344908:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.344942:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72581120, count: 4096 00000080:00200000:2.0:1789760783.344949:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72581120, bytes: 4096 00000080:00200000:2.0:1789760783.344962:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.344969:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.344971:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72581120, 72585216) 0 0 00000080:00200000:2.0:1789760783.344976:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.344992:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17720, 17720] 00020000:00200000:2.0:1789760783.345002:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72581120 chunk: [72581120, 72585216) 72585216, 4096 00020000:00200000:2.0:1789760783.345006:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.345028:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17720, 17720] 00000008:00010000:2.0:1789760783.345053:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.345059:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.345073:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72581120, 72585216) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.345088:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72581120=0x4538000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.345316:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.345319:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72585216 00000020:00010000:2.0:1789760783.345330:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.345334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.345338:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.345342:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.345736:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.345747:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.345754:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.345777:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760783.345926:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 72581120<>72585216 00000100:00100000:3.0:1789760783.346039:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565498624:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.358783:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985028 gen 3 00000008:00010000:3.0:1789760783.358929:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760783.358937:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.358948:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.358977:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760783.359009:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565498624:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.359142:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17720, 17720] 00000080:00200000:2.0:1789760783.359160:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.359166:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.359183:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72585216 i_blocks 141792 00000020:00200000:2.0:1789760783.359188:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.359198:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.359233:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72585216, count: 4096 00000080:00200000:2.0:1789760783.359241:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72585216, bytes: 4096 00000080:00200000:2.0:1789760783.359252:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.359261:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.359263:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72585216, 72589312) 0 0 00000080:00200000:2.0:1789760783.359268:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.359285:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17721, 17721] 00020000:00200000:2.0:1789760783.359294:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72585216 chunk: [72585216, 72589312) 72589312, 4096 00020000:00200000:2.0:1789760783.359298:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.359317:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17721, 17721] 00000008:00010000:2.0:1789760783.359333:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.359339:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.359351:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72585216, 72589312) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.359365:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72585216=0x4539000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.360037:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.360043:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72589312 00000020:00010000:2.0:1789760783.360054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.360058:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.360061:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.360066:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.360069:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.360078:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.360086:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.360109:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760783.360235:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 72585216<>72589312 00000100:00100000:3.0:1789760783.360311:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565498752:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.371992:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985029 gen 3 00000008:00010000:3.0:1789760783.372043:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760783.372051:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.372063:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.372130:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760783.372155:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565498752:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.372375:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17721, 17721] 00000080:00200000:2.0:1789760783.372391:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.372398:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.372412:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72589312 i_blocks 141800 00000020:00200000:2.0:1789760783.372415:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.372426:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.372459:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72589312, count: 4096 00000080:00200000:2.0:1789760783.372466:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72589312, bytes: 4096 00000080:00200000:2.0:1789760783.372492:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.372501:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.372504:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72589312, 72593408) 0 0 00000080:00200000:2.0:1789760783.372512:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.372537:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17722, 17722] 00020000:00200000:2.0:1789760783.372546:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72589312 chunk: [72589312, 72593408) 72593408, 4096 00020000:00200000:2.0:1789760783.372552:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.372572:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17722, 17722] 00000008:00010000:2.0:1789760783.372591:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.372597:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.372608:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72589312, 72593408) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.372622:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72589312=0x453a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.372746:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.372749:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72593408 00000020:00010000:2.0:1789760783.372756:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.372759:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.372761:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.372763:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.372766:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.372771:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.372776:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.372791:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.372965:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 72589312<>72593408 00000100:00100000:3.0:1789760783.373041:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565498880:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.383325:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985030 gen 3 00000008:00010000:3.0:1789760783.383382:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760783.383391:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.383403:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.383431:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760783.383464:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565498880:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.384108:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17722, 17722] 00000080:00200000:2.0:1789760783.384126:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.384133:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.384149:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72593408 i_blocks 141808 00000020:00200000:2.0:1789760783.384154:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.384164:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.384197:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72593408, count: 4096 00000080:00200000:2.0:1789760783.384204:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72593408, bytes: 4096 00000080:00200000:2.0:1789760783.384217:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.384227:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.384229:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72593408, 72597504) 0 0 00000080:00200000:2.0:1789760783.384234:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.384250:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17723, 17723] 00020000:00200000:2.0:1789760783.384259:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72593408 chunk: [72593408, 72597504) 72597504, 4096 00020000:00200000:2.0:1789760783.384264:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.384285:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17723, 17723] 00000008:00010000:2.0:1789760783.384302:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.384308:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.384320:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72593408, 72597504) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.384333:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72593408=0x453b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.384391:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.384394:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72597504 00000020:00010000:2.0:1789760783.384402:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.384406:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.384409:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.384412:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.384416:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.384423:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.384430:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.384447:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.385074:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 72593408<>72597504 00000100:00100000:2.0:1789760783.385198:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565499008:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.399903:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985031 gen 3 00000008:00010000:2.0:1789760783.399957:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760783.399966:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.399977:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.400140:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760783.400172:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565499008:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760783.400294:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17723, 17723] 00000080:00200000:3.0:1789760783.400311:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760783.400317:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760783.400334:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72597504 i_blocks 141816 00000020:00200000:3.0:1789760783.400337:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760783.400348:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760783.400488:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72597504, count: 4096 00000080:00200000:3.0:1789760783.400497:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72597504, bytes: 4096 00000080:00200000:3.0:1789760783.400509:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760783.400519:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760783.400521:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72597504, 72601600) 0 0 00000080:00200000:3.0:1789760783.400527:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760783.400544:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17724, 17724] 00020000:00200000:3.0:1789760783.400597:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72597504 chunk: [72597504, 72601600) 72601600, 4096 00020000:00200000:3.0:1789760783.400603:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760783.400624:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17724, 17724] 00000008:00010000:3.0:1789760783.400729:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760783.400734:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760783.400747:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72597504, 72601600) append 0 append lockpos 0 00000080:00200000:3.0:1789760783.400854:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72597504=0x453c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760783.400919:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760783.400922:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72601600 00000020:00010000:3.0:1789760783.400930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760783.400934:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.400938:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.400941:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760783.400945:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760783.400954:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760783.400961:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760783.400982:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760783.401188:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 72597504<>72601600 00000100:00100000:2.0:1789760783.401273:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565499136:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.414407:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985032 gen 3 00000008:00010000:2.0:1789760783.414579:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760783.414588:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.414600:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.414622:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000080:00200000:3.0:1789760783.414750:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17724, 17724] 00000080:00200000:3.0:1789760783.414769:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760783.414775:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760783.414793:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72601600 i_blocks 141824 00000020:00200000:3.0:1789760783.414797:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760783.414807:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000100:00100000:2.0:1789760783.415696:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565499136:192.168.203.147@tcp:4:dd.0 00000080:00200008:3.0:1789760783.416225:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72601600, count: 4096 00000080:00200000:3.0:1789760783.416238:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72601600, bytes: 4096 00000080:00200000:3.0:1789760783.416253:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760783.416263:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760783.416265:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72601600, 72605696) 0 0 00000080:00200000:3.0:1789760783.416271:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760783.416289:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17725, 17725] 00020000:00200000:3.0:1789760783.416299:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72601600 chunk: [72601600, 72605696) 72605696, 4096 00020000:00200000:3.0:1789760783.416304:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760783.416326:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17725, 17725] 00000008:00010000:3.0:1789760783.416345:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760783.416352:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760783.416365:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72601600, 72605696) append 0 append lockpos 0 00000080:00200000:3.0:1789760783.416379:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72601600=0x453d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760783.416742:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760783.416746:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72605696 00000020:00010000:3.0:1789760783.416756:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760783.416760:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.416763:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.416766:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760783.416769:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760783.416779:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760783.416786:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760783.416810:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760783.417180:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 72601600<>72605696 00000100:00100000:2.0:1789760783.417313:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565499264:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.428383:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985033 gen 3 00000008:00010000:2.0:1789760783.428447:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760783.428456:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.428468:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.428506:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760783.428535:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565499264:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760783.428710:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17725, 17725] 00000080:00200000:3.0:1789760783.428729:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760783.428735:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760783.428751:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72605696 i_blocks 141832 00000020:00200000:3.0:1789760783.428756:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760783.428767:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760783.428801:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72605696, count: 4096 00000080:00200000:3.0:1789760783.428808:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72605696, bytes: 4096 00000080:00200000:3.0:1789760783.428818:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760783.428827:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760783.428829:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72605696, 72609792) 0 0 00000080:00200000:3.0:1789760783.428833:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760783.428850:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17726, 17726] 00020000:00200000:3.0:1789760783.428859:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72605696 chunk: [72605696, 72609792) 72609792, 4096 00020000:00200000:3.0:1789760783.428877:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760783.428900:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17726, 17726] 00000008:00010000:3.0:1789760783.428918:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760783.428925:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760783.428936:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72605696, 72609792) append 0 append lockpos 0 00000080:00200000:3.0:1789760783.428950:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72605696=0x453e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760783.429009:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760783.429012:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72609792 00000020:00010000:3.0:1789760783.429020:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760783.429024:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.429027:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.429030:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760783.429034:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760783.429043:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760783.429049:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760783.429066:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760783.429213:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 72605696<>72609792 00000100:00100000:2.0:1789760783.429393:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760783.429410:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565499392:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.442282:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985034 gen 3 00000008:00010000:2.0:1789760783.442336:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760783.442344:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.442354:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.442376:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760783.442405:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565499392:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760783.442707:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17726, 17726] 00000080:00200000:3.0:1789760783.442727:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760783.442734:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.442848:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72609792 i_blocks 141840 00000020:00200000:2.0:1789760783.442853:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.442864:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.442902:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72609792, count: 4096 00000080:00200000:2.0:1789760783.442910:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72609792, bytes: 4096 00000080:00200000:2.0:1789760783.442921:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.442929:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.442932:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72609792, 72613888) 0 0 00000080:00200000:2.0:1789760783.442936:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.442953:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17727, 17727] 00020000:00200000:2.0:1789760783.442962:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72609792 chunk: [72609792, 72613888) 72613888, 4096 00020000:00200000:2.0:1789760783.442968:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.442988:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17727, 17727] 00000008:00010000:2.0:1789760783.443007:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.443014:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.443026:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72609792, 72613888) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.443040:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72609792=0x453f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.443100:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.443109:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72613888 00000020:00010000:2.0:1789760783.443117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.443121:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.443124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.443128:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.443131:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.443138:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.443145:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.443192:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.443316:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 72609792<>72613888 00000100:00100000:2.0:1789760783.443368:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760783.443388:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565499520:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.458011:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985035 gen 3 00000008:00010000:2.0:1789760783.458062:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760783.458072:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.458083:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.458164:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760783.458193:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565499520:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760783.460756:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17727, 17727] 00000080:00200000:3.0:1789760783.460781:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760783.460854:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.461100:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72613888 i_blocks 141848 00000020:00200000:2.0:1789760783.461107:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.461124:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.461159:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72613888, count: 4096 00000080:00200000:2.0:1789760783.461167:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72613888, bytes: 4096 00000080:00200000:2.0:1789760783.461182:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.461192:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.461194:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72613888, 72617984) 0 0 00000080:00200000:2.0:1789760783.461199:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.461215:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17728, 17728] 00020000:00200000:2.0:1789760783.461225:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72613888 chunk: [72613888, 72617984) 72617984, 4096 00020000:00200000:2.0:1789760783.461229:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.461251:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17728, 17728] 00000008:00010000:2.0:1789760783.461268:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.461273:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.461285:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72613888, 72617984) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.461300:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72613888=0x4540000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.461364:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.461367:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72617984 00000020:00010000:2.0:1789760783.461374:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.461378:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.461382:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.461386:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.461389:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.461397:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.461403:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.461466:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.461856:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 72613888<>72617984 00000100:00100000:3.0:1789760783.462123:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565499648:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.477583:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985036 gen 3 00000008:00010000:3.0:1789760783.477637:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760783.477688:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.477700:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.477727:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760783.477756:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565499648:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.477940:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17728, 17728] 00000080:00200000:2.0:1789760783.478015:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.478022:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.478037:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72617984 i_blocks 141856 00000020:00200000:2.0:1789760783.478040:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.478051:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.478084:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72617984, count: 4096 00000080:00200000:2.0:1789760783.478092:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72617984, bytes: 4096 00000080:00200000:2.0:1789760783.478103:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.478112:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.478115:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72617984, 72622080) 0 0 00000080:00200000:2.0:1789760783.478119:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.478136:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17729, 17729] 00020000:00200000:2.0:1789760783.478144:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72617984 chunk: [72617984, 72622080) 72622080, 4096 00020000:00200000:2.0:1789760783.478149:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.478168:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17729, 17729] 00000008:00010000:2.0:1789760783.478185:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.478191:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.478202:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72617984, 72622080) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.478214:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72617984=0x4541000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.478274:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.478277:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72622080 00000020:00010000:2.0:1789760783.478285:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.478289:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.478292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.478295:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.478299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.478306:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.478313:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.478329:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.478444:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 72617984<>72622080 00000100:00100000:3.0:1789760783.478582:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565499776:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.489462:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985037 gen 3 00000008:00010000:2.0:1789760783.489537:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760783.489546:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.489558:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.489581:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760783.489609:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565499776:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.489862:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17729, 17729] 00000080:00200000:1.0:1789760783.489881:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.489887:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.489913:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72622080 i_blocks 141864 00000020:00200000:1.0:1789760783.489918:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.489928:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.489964:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72622080, count: 4096 00000080:00200000:1.0:1789760783.489971:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72622080, bytes: 4096 00000080:00200000:1.0:1789760783.489981:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.489990:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.489993:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72622080, 72626176) 0 0 00000080:00200000:1.0:1789760783.489997:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.490014:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17730, 17730] 00020000:00200000:1.0:1789760783.490023:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72622080 chunk: [72622080, 72626176) 72626176, 4096 00020000:00200000:1.0:1789760783.490027:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.490046:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17730, 17730] 00000008:00010000:1.0:1789760783.490064:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.490070:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.490081:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72622080, 72626176) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.490094:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72622080=0x4542000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.490154:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.490157:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72626176 00000020:00010000:1.0:1789760783.490165:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.490169:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.490172:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.490176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.490180:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.490187:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.490194:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.490211:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760783.490350:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 72622080<>72626176 00000100:00100000:0.0:1789760783.491910:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565499904:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.505359:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985038 gen 3 00000008:00010000:0.0:1789760783.505417:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760783.505436:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.505446:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.505465:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760783.505502:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565499904:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.505600:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17730, 17730] 00000080:00200000:1.0:1789760783.505616:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.505622:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.505637:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72626176 i_blocks 141872 00000020:00200000:1.0:1789760783.505641:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.505680:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.506045:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72626176, count: 4096 00000080:00200000:1.0:1789760783.506055:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72626176, bytes: 4096 00000080:00200000:1.0:1789760783.506066:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.506075:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.506078:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72626176, 72630272) 0 0 00000080:00200000:1.0:1789760783.506082:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.506099:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17731, 17731] 00020000:00200000:1.0:1789760783.506108:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72626176 chunk: [72626176, 72630272) 72630272, 4096 00020000:00200000:1.0:1789760783.506112:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.506132:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17731, 17731] 00000008:00010000:1.0:1789760783.506153:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.506159:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.506171:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72626176, 72630272) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.506185:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72626176=0x4543000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.506245:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.506248:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72630272 00000020:00010000:1.0:1789760783.506256:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.506260:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.506264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.506267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.506271:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.506280:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.506287:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.506308:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760783.506444:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 72626176<>72630272 00000100:00100000:0.0:1789760783.506522:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760783.506542:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565500032:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.519733:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985039 gen 3 00000008:00010000:0.0:1789760783.519789:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760783.519798:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.519810:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.519837:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760783.519867:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565500032:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.520059:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17731, 17731] 00000080:00200000:1.0:1789760783.520077:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.520083:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.520102:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72630272 i_blocks 141880 00000020:00200000:1.0:1789760783.520106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.520118:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.520155:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72630272, count: 4096 00000080:00200000:1.0:1789760783.520164:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72630272, bytes: 4096 00000080:00200000:1.0:1789760783.520179:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.520188:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.520191:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72630272, 72634368) 0 0 00000080:00200000:1.0:1789760783.520195:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.520213:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17732, 17732] 00020000:00200000:1.0:1789760783.520223:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72630272 chunk: [72630272, 72634368) 72634368, 4096 00020000:00200000:1.0:1789760783.520227:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.520249:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17732, 17732] 00000008:00010000:1.0:1789760783.520267:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.520272:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.520285:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72630272, 72634368) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.520298:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72630272=0x4544000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.520353:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.520356:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72634368 00000020:00010000:1.0:1789760783.520364:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.520368:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.520371:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.520374:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.520378:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.520385:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.520392:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.520424:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760783.520587:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 72630272<>72634368 00000100:00100000:1.0:1789760783.520634:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760783.520698:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565500160:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.530384:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985040 gen 3 00000008:00010000:0.0:1789760783.530454:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760783.530464:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.530486:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.530499:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760783.530529:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565500160:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.530837:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17732, 17732] 00000080:00200000:1.0:1789760783.530855:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.530861:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.530877:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72634368 i_blocks 141888 00000020:00200000:1.0:1789760783.530881:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.530890:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.531028:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72634368, count: 4096 00000080:00200000:1.0:1789760783.531036:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72634368, bytes: 4096 00000080:00200000:1.0:1789760783.531047:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.531056:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.531058:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72634368, 72638464) 0 0 00000080:00200000:1.0:1789760783.531063:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.531079:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17733, 17733] 00020000:00200000:1.0:1789760783.531087:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72634368 chunk: [72634368, 72638464) 72638464, 4096 00020000:00200000:1.0:1789760783.531091:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.531110:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17733, 17733] 00000008:00010000:1.0:1789760783.531127:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.531132:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.531143:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72634368, 72638464) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.531156:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72634368=0x4545000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.531209:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.531211:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72638464 00000020:00010000:1.0:1789760783.531218:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.531222:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.531225:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.531229:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.531232:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.531238:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.531244:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.531258:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760783.531368:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 72634368<>72638464 00000100:00100000:0.0:1789760783.531541:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565500288:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.540079:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985041 gen 3 00000008:00010000:0.0:1789760783.540127:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760783.540135:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.540146:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.540214:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760783.540241:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565500288:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.542046:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17733, 17733] 00000080:00200000:2.0:1789760783.542074:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.542081:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.542100:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72638464 i_blocks 141896 00000020:00200000:2.0:1789760783.542104:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.542131:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.542166:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72638464, count: 4096 00000080:00200000:2.0:1789760783.542175:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72638464, bytes: 4096 00000080:00200000:2.0:1789760783.542190:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.542199:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.542201:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72638464, 72642560) 0 0 00000080:00200000:2.0:1789760783.542205:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.542222:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17734, 17734] 00020000:00200000:2.0:1789760783.542230:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72638464 chunk: [72638464, 72642560) 72642560, 4096 00020000:00200000:2.0:1789760783.542235:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.542256:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17734, 17734] 00000008:00010000:2.0:1789760783.542274:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.542280:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.542291:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72638464, 72642560) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.542306:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72638464=0x4546000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.542385:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.542388:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72642560 00000020:00010000:2.0:1789760783.542397:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.542401:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.542404:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.542789:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.542793:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.542804:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.542813:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.542855:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.542995:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 72638464<>72642560 00000100:00100000:3.0:1789760783.543081:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565500416:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.555082:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985042 gen 3 00000008:00010000:3.0:1789760783.555128:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760783.555135:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.555146:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.555231:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760783.555262:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565500416:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.558578:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17734, 17734] 00000080:00200000:2.0:1789760783.558607:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.558615:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.558633:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72642560 i_blocks 141904 00000020:00200000:2.0:1789760783.558639:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.558670:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.559129:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72642560, count: 4096 00000080:00200000:2.0:1789760783.559139:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72642560, bytes: 4096 00000080:00200000:2.0:1789760783.559155:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.559164:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.559166:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72642560, 72646656) 0 0 00000080:00200000:2.0:1789760783.559171:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.559187:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17735, 17735] 00020000:00200000:2.0:1789760783.559197:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72642560 chunk: [72642560, 72646656) 72646656, 4096 00020000:00200000:2.0:1789760783.559202:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.559224:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17735, 17735] 00000008:00010000:2.0:1789760783.559244:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.559251:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.559262:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72642560, 72646656) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.559276:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72642560=0x4547000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.559345:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.559348:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72646656 00000020:00010000:2.0:1789760783.559356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.559361:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.559364:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.559367:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.559370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.559379:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.559386:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.559790:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f8441893c0 00000008:00100000:2.0:1789760783.559919:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 72642560<>72646656 00000100:00100000:3.0:1789760783.560561:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565500544:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.582938:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985043 gen 3 00000008:00010000:3.0:1789760783.582994:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760783.583002:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.583010:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.583028:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760783.583050:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565500544:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.584082:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17735, 17735] 00000080:00200000:2.0:1789760783.584106:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.584112:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.584129:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72646656 i_blocks 141912 00000020:00200000:2.0:1789760783.584135:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.584147:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.584204:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72646656, count: 4096 00000080:00200000:2.0:1789760783.584215:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72646656, bytes: 4096 00000080:00200000:2.0:1789760783.584409:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.584424:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.584429:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72646656, 72650752) 0 0 00000080:00200000:2.0:1789760783.584435:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.584466:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17736, 17736] 00020000:00200000:2.0:1789760783.584483:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72646656 chunk: [72646656, 72650752) 72650752, 4096 00020000:00200000:2.0:1789760783.584489:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.584513:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17736, 17736] 00000008:00010000:2.0:1789760783.584532:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.584538:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.584550:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72646656, 72650752) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.584574:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72646656=0x4548000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.584749:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.584753:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72650752 00000020:00010000:2.0:1789760783.584762:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.584767:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.584769:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.584773:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.584776:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.584785:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.584792:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.584812:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f8441893c0 00000008:00100000:2.0:1789760783.584941:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 72646656<>72650752 00000100:00100000:3.0:1789760783.586080:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565500672:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.595818:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985044 gen 3 00000008:00010000:3.0:1789760783.595868:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760783.595877:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.595888:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.595910:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760783.595940:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565500672:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.596112:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17736, 17736] 00000080:00200000:2.0:1789760783.596130:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.596137:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.596151:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72650752 i_blocks 141920 00000020:00200000:2.0:1789760783.596156:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.596166:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.596200:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72650752, count: 4096 00000080:00200000:2.0:1789760783.596208:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72650752, bytes: 4096 00000080:00200000:2.0:1789760783.596218:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.596405:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.596462:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72650752, 72654848) 0 0 00000080:00200000:2.0:1789760783.596467:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.596494:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17737, 17737] 00020000:00200000:2.0:1789760783.596502:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72650752 chunk: [72650752, 72654848) 72654848, 4096 00020000:00200000:2.0:1789760783.596506:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.596525:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17737, 17737] 00000008:00010000:2.0:1789760783.596543:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.596549:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.596561:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72650752, 72654848) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.597254:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72650752=0x4549000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.597335:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.597338:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72654848 00000020:00010000:2.0:1789760783.597353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.597359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.597362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.597366:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.597368:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.597378:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.597385:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.597548:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.597785:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 72650752<>72654848 00000100:00100000:3.0:1789760783.598714:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760783.598743:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565500800:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.613834:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985045 gen 3 00000008:00010000:3.0:1789760783.613887:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760783.613895:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.613907:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.613933:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760783.613964:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565500800:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.614515:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17737, 17737] 00000080:00200000:2.0:1789760783.614534:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.614539:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.614555:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72654848 i_blocks 141928 00000020:00200000:2.0:1789760783.614560:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.615292:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.615330:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72654848, count: 4096 00000080:00200000:2.0:1789760783.615340:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72654848, bytes: 4096 00000080:00200000:2.0:1789760783.615363:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.615373:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.615376:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72654848, 72658944) 0 0 00000080:00200000:2.0:1789760783.615380:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.615398:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17738, 17738] 00020000:00200000:2.0:1789760783.615481:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72654848 chunk: [72654848, 72658944) 72658944, 4096 00020000:00200000:2.0:1789760783.615487:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.615511:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17738, 17738] 00000008:00010000:2.0:1789760783.615528:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.615534:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.615547:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72654848, 72658944) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.615561:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72654848=0x454a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.615788:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.615792:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72658944 00000020:00010000:2.0:1789760783.615800:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.615805:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.615807:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.615811:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.615814:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.615823:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.615830:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.615856:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.615980:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 72654848<>72658944 00000100:00100000:2.0:1789760783.616036:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760783.616059:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565500928:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.628118:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985046 gen 3 00000008:00010000:2.0:1789760783.628173:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760783.628182:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.628193:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.628217:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760783.628245:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565500928:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.628718:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17738, 17738] 00000080:00200000:1.0:1789760783.628734:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.628741:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.628756:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72658944 i_blocks 141936 00000020:00200000:1.0:1789760783.628759:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.628770:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.628804:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72658944, count: 4096 00000080:00200000:1.0:1789760783.628811:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72658944, bytes: 4096 00000080:00200000:1.0:1789760783.628890:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.628899:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.628902:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72658944, 72663040) 0 0 00000080:00200000:1.0:1789760783.628906:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.628922:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17739, 17739] 00020000:00200000:1.0:1789760783.628932:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72658944 chunk: [72658944, 72663040) 72663040, 4096 00020000:00200000:1.0:1789760783.628936:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.628955:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17739, 17739] 00000008:00010000:1.0:1789760783.628972:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.628978:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.628989:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72658944, 72663040) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.629004:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72658944=0x454b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.629067:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.629070:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72663040 00000020:00010000:1.0:1789760783.629078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.629082:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.629086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.629090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.629093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.629101:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.629107:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.629125:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760783.629250:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 72658944<>72663040 00000100:00100000:0.0:1789760783.629334:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565501056:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.641719:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985047 gen 3 00000008:00010000:0.0:1789760783.641773:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760783.641783:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.641794:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.641820:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760783.641850:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565501056:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.641997:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17739, 17739] 00000080:00200000:1.0:1789760783.642013:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.642019:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.642035:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72663040 i_blocks 141944 00000020:00200000:1.0:1789760783.642040:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.642050:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.642084:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72663040, count: 4096 00000080:00200000:1.0:1789760783.642091:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72663040, bytes: 4096 00000080:00200000:1.0:1789760783.642105:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.642113:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.642116:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72663040, 72667136) 0 0 00000080:00200000:1.0:1789760783.642121:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.642137:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17740, 17740] 00020000:00200000:1.0:1789760783.642146:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72663040 chunk: [72663040, 72667136) 72667136, 4096 00020000:00200000:1.0:1789760783.642150:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.642170:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17740, 17740] 00000008:00010000:1.0:1789760783.642188:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.642195:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.642207:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72663040, 72667136) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.642221:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72663040=0x454c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.642279:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.642282:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72667136 00000020:00010000:1.0:1789760783.642290:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.642293:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.642297:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.642300:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.642303:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.642310:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.642317:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.642333:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760783.642455:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 72663040<>72667136 00000100:00100000:0.0:1789760783.642722:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760783.642741:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565501184:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.661846:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985048 gen 3 00000008:00010000:0.0:1789760783.661894:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760783.661903:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.661913:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.661943:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760783.661972:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565501184:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.662095:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17740, 17740] 00000080:00200000:2.0:1789760783.662113:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.662119:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.662134:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72667136 i_blocks 141952 00000020:00200000:2.0:1789760783.662138:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.662147:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.662178:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72667136, count: 4096 00000080:00200000:2.0:1789760783.662185:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72667136, bytes: 4096 00000080:00200000:2.0:1789760783.662195:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.662204:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.662206:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72667136, 72671232) 0 0 00000080:00200000:2.0:1789760783.662210:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.662226:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17741, 17741] 00020000:00200000:2.0:1789760783.662246:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72667136 chunk: [72667136, 72671232) 72671232, 4096 00020000:00200000:2.0:1789760783.662251:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.662270:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17741, 17741] 00000008:00010000:2.0:1789760783.662288:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.662293:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.662304:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72667136, 72671232) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.662317:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72667136=0x454d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.662373:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.662376:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72671232 00000020:00010000:2.0:1789760783.662383:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.662387:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.662390:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.662393:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.662396:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.662404:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.662411:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.662426:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760783.663859:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 72667136<>72671232 00000100:00100000:3.0:1789760783.664621:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760783.664877:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565501312:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.678246:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985049 gen 3 00000008:00010000:3.0:1789760783.678294:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760783.678302:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.678311:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.678332:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760783.678363:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565501312:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.678559:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17741, 17741] 00000080:00200000:2.0:1789760783.678578:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.678584:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.678600:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72671232 i_blocks 141960 00000020:00200000:2.0:1789760783.678604:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.678613:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.678691:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72671232, count: 4096 00000080:00200000:2.0:1789760783.678699:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72671232, bytes: 4096 00000080:00200000:2.0:1789760783.678709:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.678717:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.678720:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72671232, 72675328) 0 0 00000080:00200000:2.0:1789760783.678725:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.678743:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17742, 17742] 00020000:00200000:2.0:1789760783.678753:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72671232 chunk: [72671232, 72675328) 72675328, 4096 00020000:00200000:2.0:1789760783.678757:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.678778:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17742, 17742] 00000008:00010000:2.0:1789760783.678796:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.678802:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.678813:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72671232, 72675328) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.678827:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72671232=0x454e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.678890:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.678893:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72675328 00000020:00010000:2.0:1789760783.678901:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.678905:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.678908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.678911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.678914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.678922:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.678929:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.678944:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f8441893c0 00000008:00100000:2.0:1789760783.679248:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 72671232<>72675328 00000100:00100000:3.0:1789760783.679307:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565501440:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.693238:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985050 gen 3 00000008:00010000:3.0:1789760783.693288:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760783.693296:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.693306:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.693332:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760783.693366:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565501440:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.693490:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17742, 17742] 00000080:00200000:1.0:1789760783.693507:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.693513:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.693530:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72675328 i_blocks 141968 00000020:00200000:1.0:1789760783.693533:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.693543:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.693575:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72675328, count: 4096 00000080:00200000:1.0:1789760783.693582:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72675328, bytes: 4096 00000080:00200000:1.0:1789760783.693593:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.693602:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.693605:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72675328, 72679424) 0 0 00000080:00200000:1.0:1789760783.693609:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.693625:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17743, 17743] 00020000:00200000:1.0:1789760783.693634:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72675328 chunk: [72675328, 72679424) 72679424, 4096 00020000:00200000:1.0:1789760783.693639:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.693697:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17743, 17743] 00000008:00010000:1.0:1789760783.693717:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.693724:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.693736:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72675328, 72679424) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.693752:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72675328=0x454f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.693809:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.693812:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72679424 00000020:00010000:1.0:1789760783.693820:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.693823:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.693827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.693830:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.693833:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.693841:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.693847:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.693863:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760783.693985:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 72675328<>72679424 00000100:00100000:0.0:1789760783.694210:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565501568:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.710704:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985051 gen 3 00000008:00010000:0.0:1789760783.710754:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760783.710992:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.711005:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.711023:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760783.711054:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565501568:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.711513:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17743, 17743] 00000080:00200000:1.0:1789760783.711530:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.711536:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.711553:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72679424 i_blocks 141976 00000020:00200000:1.0:1789760783.711557:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.711567:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.711603:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72679424, count: 4096 00000080:00200000:1.0:1789760783.711610:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72679424, bytes: 4096 00000080:00200000:1.0:1789760783.711622:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.711630:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.711633:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72679424, 72683520) 0 0 00000080:00200000:1.0:1789760783.711637:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.711684:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17744, 17744] 00020000:00200000:1.0:1789760783.711695:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72679424 chunk: [72679424, 72683520) 72683520, 4096 00020000:00200000:1.0:1789760783.711700:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.711722:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17744, 17744] 00000008:00010000:1.0:1789760783.711740:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.711746:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.711758:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72679424, 72683520) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.712207:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72679424=0x4550000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.712273:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.712276:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72683520 00000020:00010000:1.0:1789760783.712284:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.712287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.712291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.712294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.712297:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.712306:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.712313:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.712334:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760783.712610:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 72679424<>72683520 00000100:00100000:0.0:1789760783.712702:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565501696:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.730978:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985052 gen 3 00000008:00010000:0.0:1789760783.731024:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760783.731032:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.731041:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.731065:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760783.731091:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565501696:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760783.731375:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17744, 17744] 00000080:00200000:3.0:1789760783.731425:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760783.731432:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760783.731448:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72683520 i_blocks 141984 00000020:00200000:3.0:1789760783.731452:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760783.731461:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760783.731518:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72683520, count: 4096 00000080:00200000:3.0:1789760783.731527:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72683520, bytes: 4096 00000080:00200000:3.0:1789760783.731540:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760783.731548:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760783.731550:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72683520, 72687616) 0 0 00000080:00200000:3.0:1789760783.731554:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760783.731571:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17745, 17745] 00020000:00200000:3.0:1789760783.731580:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72683520 chunk: [72683520, 72687616) 72687616, 4096 00020000:00200000:3.0:1789760783.731585:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760783.731607:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17745, 17745] 00000008:00010000:3.0:1789760783.731623:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760783.731629:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760783.731641:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72683520, 72687616) append 0 append lockpos 0 00000080:00200000:3.0:1789760783.731689:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72683520=0x4551000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760783.731750:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760783.731752:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72687616 00000020:00010000:3.0:1789760783.731761:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760783.731764:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.731767:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.731771:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760783.731774:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760783.731782:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760783.731789:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760783.731806:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760783.731932:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 72683520<>72687616 00000100:00100000:2.0:1789760783.732085:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565501824:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.745959:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985053 gen 3 00000008:00010000:2.0:1789760783.746010:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760783.746019:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.746031:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.746094:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760783.746126:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565501824:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760783.746304:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17745, 17745] 00000080:00200000:3.0:1789760783.746322:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760783.746328:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.746554:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72687616 i_blocks 141992 00000020:00200000:2.0:1789760783.746559:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.746569:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.746604:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72687616, count: 4096 00000080:00200000:2.0:1789760783.746612:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72687616, bytes: 4096 00000080:00200000:2.0:1789760783.746624:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.746633:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.746636:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72687616, 72691712) 0 0 00000080:00200000:2.0:1789760783.746641:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.746684:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17746, 17746] 00020000:00200000:2.0:1789760783.746693:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72687616 chunk: [72687616, 72691712) 72691712, 4096 00020000:00200000:2.0:1789760783.746698:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.746719:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17746, 17746] 00000008:00010000:2.0:1789760783.746737:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.746742:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.746754:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72687616, 72691712) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.746769:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72687616=0x4552000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.746964:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.746968:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72691712 00000020:00010000:2.0:1789760783.746977:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.746981:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.746984:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.746988:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.746991:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.746998:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.747005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.747051:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760783.747232:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 72687616<>72691712 00000100:00100000:3.0:1789760783.747311:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565501952:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.762562:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985054 gen 3 00000008:00010000:2.0:1789760783.762611:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760783.762619:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.762630:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.762671:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760783.762701:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565501952:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.762857:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17746, 17746] 00000080:00200000:0.0:1789760783.762875:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.762882:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.762899:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72691712 i_blocks 142000 00000020:00200000:0.0:1789760783.762902:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.762913:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.762948:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72691712, count: 4096 00000080:00200000:0.0:1789760783.762954:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72691712, bytes: 4096 00000080:00200000:0.0:1789760783.762963:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.762970:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.762972:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72691712, 72695808) 0 0 00000080:00200000:0.0:1789760783.762976:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.762990:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17747, 17747] 00020000:00200000:0.0:1789760783.762997:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72691712 chunk: [72691712, 72695808) 72695808, 4096 00020000:00200000:0.0:1789760783.763001:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.763017:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17747, 17747] 00000008:00010000:0.0:1789760783.763033:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.763038:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.763051:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72691712, 72695808) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.763065:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72691712=0x4553000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.763130:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.763133:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72695808 00000020:00010000:0.0:1789760783.763141:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.763144:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.763147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.763151:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.763154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.763162:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.763168:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760783.763184:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760783.763380:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72691712<>72695808 00000100:00100000:0.0:1789760783.763751:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760783.763771:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565502080:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.772349:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985055 gen 3 00000008:00010000:0.0:1789760783.772398:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760783.772476:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.772538:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.772565:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760783.772596:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565502080:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.772747:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17747, 17747] 00000080:00200000:2.0:1789760783.772763:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.772769:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.772784:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72695808 i_blocks 142008 00000020:00200000:2.0:1789760783.772787:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.772800:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.772898:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72695808, count: 4096 00000080:00200000:2.0:1789760783.772907:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72695808, bytes: 4096 00000080:00200000:2.0:1789760783.772918:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.772928:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.772930:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72695808, 72699904) 0 0 00000080:00200000:2.0:1789760783.772935:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.772952:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17748, 17748] 00020000:00200000:2.0:1789760783.772961:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72695808 chunk: [72695808, 72699904) 72699904, 4096 00020000:00200000:2.0:1789760783.772966:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.772985:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17748, 17748] 00000008:00010000:2.0:1789760783.773002:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.773008:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.773021:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72695808, 72699904) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.773035:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72695808=0x4554000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.773089:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.773092:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72699904 00000020:00010000:2.0:1789760783.773099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.773103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.773106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.773110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.773113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.773120:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.773126:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.773142:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760783.773329:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 72695808<>72699904 00000100:00100000:3.0:1789760783.773405:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565502208:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.782803:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985056 gen 3 00000008:00010000:2.0:1789760783.782929:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760783.782938:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.782948:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.783024:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760783.783054:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565502208:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.783547:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17748, 17748] 00000080:00200000:0.0:1789760783.783564:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.783570:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.783586:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72699904 i_blocks 142016 00000020:00200000:0.0:1789760783.783590:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.783600:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.783635:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72699904, count: 4096 00000080:00200000:0.0:1789760783.783642:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72699904, bytes: 4096 00000080:00200000:0.0:1789760783.783684:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.783693:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.783696:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72699904, 72704000) 0 0 00000080:00200000:0.0:1789760783.783701:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.783718:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17749, 17749] 00020000:00200000:0.0:1789760783.783727:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72699904 chunk: [72699904, 72704000) 72704000, 4096 00020000:00200000:0.0:1789760783.783732:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.783751:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17749, 17749] 00000008:00010000:0.0:1789760783.783769:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.783775:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.783785:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72699904, 72704000) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.783799:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72699904=0x4555000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.783867:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.783870:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72704000 00000020:00010000:0.0:1789760783.783878:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.783882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.783885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.783889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.783892:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.783901:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.783907:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760783.783928:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760783.784049:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72699904<>72704000 00000100:00100000:1.0:1789760783.784237:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760783.784257:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565502336:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.792635:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985057 gen 3 00000008:00010000:0.0:1789760783.792717:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760783.792725:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.792736:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.792760:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760783.792789:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565502336:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.793197:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17749, 17749] 00000080:00200000:2.0:1789760783.793214:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.793220:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.793246:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72704000 i_blocks 142024 00000020:00200000:2.0:1789760783.793250:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.793262:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.793296:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72704000, count: 4096 00000080:00200000:2.0:1789760783.793304:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72704000, bytes: 4096 00000080:00200000:2.0:1789760783.793315:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.793324:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.793326:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72704000, 72708096) 0 0 00000080:00200000:2.0:1789760783.793331:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.793349:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17750, 17750] 00020000:00200000:2.0:1789760783.793358:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72704000 chunk: [72704000, 72708096) 72708096, 4096 00020000:00200000:2.0:1789760783.793362:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.793383:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17750, 17750] 00000008:00010000:2.0:1789760783.793402:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.793574:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.793589:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72704000, 72708096) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.793608:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72704000=0x4556000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.794083:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.794088:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72708096 00000020:00010000:2.0:1789760783.794098:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.794103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.794106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.794110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.794113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.794122:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.794129:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.794153:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760783.794351:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 72704000<>72708096 00000100:00100000:3.0:1789760783.794564:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565502464:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.806266:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985058 gen 3 00000008:00010000:2.0:1789760783.806318:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760783.806327:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.806338:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.806369:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760783.806403:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565502464:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.806513:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17750, 17750] 00000080:00200000:0.0:1789760783.806531:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.806537:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.806551:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72708096 i_blocks 142032 00000020:00200000:0.0:1789760783.806555:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.806565:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.806606:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72708096, count: 4096 00000080:00200000:0.0:1789760783.806614:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72708096, bytes: 4096 00000080:00200000:0.0:1789760783.806627:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.806635:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.806638:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72708096, 72712192) 0 0 00000080:00200000:0.0:1789760783.806674:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.806691:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17751, 17751] 00020000:00200000:0.0:1789760783.806699:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72708096 chunk: [72708096, 72712192) 72712192, 4096 00020000:00200000:0.0:1789760783.806703:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.806724:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17751, 17751] 00000008:00010000:0.0:1789760783.806740:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.806747:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.806758:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72708096, 72712192) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.806771:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72708096=0x4557000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.806823:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.806826:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72712192 00000020:00010000:0.0:1789760783.806834:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.806837:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.806840:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.806844:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.806847:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.806855:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.806862:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.807058:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:2.0:1789760783.807182:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 72708096<>72712192 00000100:00100000:3.0:1789760783.807254:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760783.808639:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565502592:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.821896:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985059 gen 3 00000008:00010000:3.0:1789760783.821957:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:3.0:1789760783.821967:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.821980:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.822003:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:3.0:1789760783.822034:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565502592:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.822269:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17751, 17751] 00000080:00200000:0.0:1789760783.822287:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.822292:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.822309:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72712192 i_blocks 142040 00000020:00200000:0.0:1789760783.822313:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.822324:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.822360:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72712192, count: 4096 00000080:00200000:0.0:1789760783.822367:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72712192, bytes: 4096 00000080:00200000:0.0:1789760783.822377:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.822386:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.822389:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72712192, 72716288) 0 0 00000080:00200000:0.0:1789760783.822393:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.822421:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17752, 17752] 00020000:00200000:0.0:1789760783.822431:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72712192 chunk: [72712192, 72716288) 72716288, 4096 00020000:00200000:0.0:1789760783.822436:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.822455:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17752, 17752] 00000008:00010000:0.0:1789760783.822482:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.822497:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.822509:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72712192, 72716288) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.822524:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72712192=0x4558000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.822581:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.822584:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72716288 00000020:00010000:0.0:1789760783.822592:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.822596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.822599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.822602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.822605:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.822613:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.822621:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760783.822640:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760783.823075:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 72712192<>72716288 00000100:00100000:1.0:1789760783.823202:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565502720:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760783.836377:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985060 gen 3 00000008:00010000:1.0:1789760783.836430:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760783.836439:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760783.836449:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760783.836546:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760783.836575:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565502720:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.841125:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17752, 17752] 00000080:00200000:2.0:1789760783.841155:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.841163:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.841181:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72716288 i_blocks 142048 00000020:00200000:2.0:1789760783.841187:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.841208:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.841245:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72716288, count: 4096 00000080:00200000:2.0:1789760783.841254:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72716288, bytes: 4096 00000080:00200000:2.0:1789760783.841270:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.841278:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.841281:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72716288, 72720384) 0 0 00000080:00200000:2.0:1789760783.841286:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.841303:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17753, 17753] 00020000:00200000:2.0:1789760783.841313:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72716288 chunk: [72716288, 72720384) 72720384, 4096 00020000:00200000:2.0:1789760783.841318:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.841342:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17753, 17753] 00000008:00010000:2.0:1789760783.841363:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.841369:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.841383:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72716288, 72720384) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.841398:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72716288=0x4559000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.841481:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.841484:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72720384 00000020:00010000:2.0:1789760783.841492:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.841495:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.841499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.841503:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.841506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.841515:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.841522:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.841550:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760783.841711:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 72716288<>72720384 00000100:00100000:2.0:1789760783.841769:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760783.841795:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565502848:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.858003:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985061 gen 3 00000008:00010000:2.0:1789760783.858073:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760783.858082:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.858094:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.858121:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760783.858154:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565502848:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760783.858177:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17753, 17753] 00000080:00200000:3.0:1789760783.858195:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760783.858200:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760783.858216:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72720384 i_blocks 142056 00000020:00200000:3.0:1789760783.858220:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760783.858231:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760783.858297:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72720384, count: 4096 00000080:00200000:3.0:1789760783.858306:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72720384, bytes: 4096 00000080:00200000:3.0:1789760783.858321:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760783.858330:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760783.858332:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72720384, 72724480) 0 0 00000080:00200000:3.0:1789760783.858337:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760783.858356:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17754, 17754] 00020000:00200000:3.0:1789760783.858366:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72720384 chunk: [72720384, 72724480) 72724480, 4096 00020000:00200000:3.0:1789760783.858371:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760783.858392:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17754, 17754] 00000008:00010000:3.0:1789760783.858411:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760783.858417:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760783.858430:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72720384, 72724480) append 0 append lockpos 0 00000080:00200000:3.0:1789760783.858445:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72720384=0x455a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760783.858518:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760783.858521:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72724480 00000020:00010000:3.0:1789760783.858528:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760783.858532:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.858535:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.858538:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760783.858541:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760783.858549:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760783.858556:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760783.858573:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760783.858722:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 72720384<>72724480 00000100:00100000:3.0:1789760783.858776:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760783.858792:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565502976:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.872347:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985062 gen 3 00000008:00010000:3.0:1789760783.872402:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:3.0:1789760783.872919:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.872934:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.873000:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:3.0:1789760783.873034:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565502976:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.873594:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17754, 17754] 00000080:00200000:0.0:1789760783.873613:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.873620:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.873636:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72724480 i_blocks 142064 00000020:00200000:0.0:1789760783.873641:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.873682:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.873716:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72724480, count: 4096 00000080:00200000:0.0:1789760783.873724:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72724480, bytes: 4096 00000080:00200000:0.0:1789760783.873737:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.873744:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.873747:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72724480, 72728576) 0 0 00000080:00200000:0.0:1789760783.873751:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.873768:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17755, 17755] 00020000:00200000:0.0:1789760783.873777:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72724480 chunk: [72724480, 72728576) 72728576, 4096 00020000:00200000:0.0:1789760783.873781:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.873801:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17755, 17755] 00000008:00010000:0.0:1789760783.873889:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.873896:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.873908:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72724480, 72728576) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.873923:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72724480=0x455b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.873983:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.873986:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72728576 00000020:00010000:0.0:1789760783.873995:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.873999:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.874002:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.874005:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.874008:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.874016:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.874023:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760783.874044:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760783.874162:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72724480<>72728576 00000100:00100000:1.0:1789760783.874255:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565503104:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.888634:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985063 gen 3 00000008:00010000:0.0:1789760783.888733:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760783.888741:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.888753:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.888779:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760783.888808:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565503104:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760783.889694:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17755, 17755] 00000080:00200000:3.0:1789760783.889716:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760783.889723:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760783.889987:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72728576 i_blocks 142072 00000020:00200000:3.0:1789760783.889993:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760783.890010:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760783.890045:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72728576, count: 4096 00000080:00200000:3.0:1789760783.890054:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72728576, bytes: 4096 00000080:00200000:3.0:1789760783.890068:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760783.890077:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760783.890080:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72728576, 72732672) 0 0 00000080:00200000:3.0:1789760783.890084:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760783.890102:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17756, 17756] 00020000:00200000:3.0:1789760783.890112:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72728576 chunk: [72728576, 72732672) 72732672, 4096 00020000:00200000:3.0:1789760783.890117:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760783.890139:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17756, 17756] 00000008:00010000:3.0:1789760783.890157:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760783.890164:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760783.890175:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72728576, 72732672) append 0 append lockpos 0 00000080:00200000:3.0:1789760783.890189:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72728576=0x455c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760783.890255:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760783.890258:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72732672 00000020:00010000:3.0:1789760783.890266:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760783.890269:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.890273:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.890276:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760783.890279:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760783.890288:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760783.890295:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760783.890317:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760783.891687:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 72728576<>72732672 00000100:00100000:2.0:1789760783.892045:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565503232:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760783.898937:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760783.904878:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985064 gen 3 00000008:00010000:2.0:1789760783.904937:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760783.904946:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.904958:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.904985:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760783.905019:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565503232:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760783.905141:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17756, 17756] 00000080:00200000:3.0:1789760783.905158:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760783.905164:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760783.905180:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72732672 i_blocks 142080 00000020:00200000:3.0:1789760783.905184:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760783.905193:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760783.905227:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72732672, count: 4096 00000080:00200000:3.0:1789760783.905235:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72732672, bytes: 4096 00000080:00200000:3.0:1789760783.905248:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760783.905257:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760783.905259:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72732672, 72736768) 0 0 00000080:00200000:3.0:1789760783.905263:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760783.905281:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17757, 17757] 00020000:00200000:3.0:1789760783.905290:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72732672 chunk: [72732672, 72736768) 72736768, 4096 00020000:00200000:3.0:1789760783.905294:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760783.905314:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17757, 17757] 00000008:00010000:3.0:1789760783.905334:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760783.905340:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760783.905353:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72732672, 72736768) append 0 append lockpos 0 00000080:00200000:3.0:1789760783.905366:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72732672=0x455d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760783.905523:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760783.905526:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72736768 00000020:00010000:3.0:1789760783.905536:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760783.905540:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.905543:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760783.905546:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760783.905605:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760783.905615:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760783.905622:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760783.905641:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760783.905930:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 72732672<>72736768 00000100:00100000:2.0:1789760783.905992:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565503360:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760783.918178:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985065 gen 3 00000008:00010000:3.0:1789760783.918227:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:3.0:1789760783.918235:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760783.918246:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760783.918281:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:3.0:1789760783.918313:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565503360:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.918426:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17757, 17757] 00000080:00200000:0.0:1789760783.918445:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.918451:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.918477:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72736768 i_blocks 142088 00000020:00200000:0.0:1789760783.918481:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.918492:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.918529:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72736768, count: 4096 00000080:00200000:0.0:1789760783.918536:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72736768, bytes: 4096 00000080:00200000:0.0:1789760783.918549:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.918558:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.918560:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72736768, 72740864) 0 0 00000080:00200000:0.0:1789760783.918565:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.918584:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17758, 17758] 00020000:00200000:0.0:1789760783.918594:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72736768 chunk: [72736768, 72740864) 72740864, 4096 00020000:00200000:0.0:1789760783.918598:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.918624:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17758, 17758] 00000008:00010000:0.0:1789760783.918681:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.918688:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.918701:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72736768, 72740864) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.918716:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72736768=0x455e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.918776:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.918779:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72740864 00000020:00010000:0.0:1789760783.918787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.918791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.918794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.918798:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.918801:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.918810:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.918817:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760783.918834:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760783.918962:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72736768<>72740864 00000100:00100000:0.0:1789760783.919015:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565503488:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.930009:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985066 gen 3 00000008:00010000:0.0:1789760783.930064:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760783.930072:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.930084:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.930101:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760783.930135:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565503488:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.930766:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17758, 17758] 00000080:00200000:1.0:1789760783.930785:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.930791:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.930808:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72740864 i_blocks 142096 00000020:00200000:1.0:1789760783.930884:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.930899:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.930937:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72740864, count: 4096 00000080:00200000:1.0:1789760783.930944:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72740864, bytes: 4096 00000080:00200000:1.0:1789760783.930959:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.930969:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.930972:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72740864, 72744960) 0 0 00000080:00200000:1.0:1789760783.930977:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.930995:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17759, 17759] 00020000:00200000:1.0:1789760783.931003:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72740864 chunk: [72740864, 72744960) 72744960, 4096 00020000:00200000:1.0:1789760783.931008:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.931029:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17759, 17759] 00000008:00010000:1.0:1789760783.931047:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.931053:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.931065:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72740864, 72744960) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.931079:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72740864=0x455f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.931143:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.931146:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72744960 00000020:00010000:1.0:1789760783.931154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.931157:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.931160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.931164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.931167:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.931174:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.931181:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.931201:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760783.932189:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 72740864<>72744960 00000100:00100000:0.0:1789760783.932369:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760783.932395:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565503616:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.942609:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985067 gen 3 00000008:00010000:0.0:1789760783.942714:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760783.942723:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.942734:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.942745:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760783.942774:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565503616:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760783.943119:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17759, 17759] 00000080:00200000:1.0:1789760783.943135:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760783.943141:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760783.943157:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72744960 i_blocks 142104 00000020:00200000:1.0:1789760783.943161:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760783.943171:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760783.943207:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72744960, count: 4096 00000080:00200000:1.0:1789760783.943215:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72744960, bytes: 4096 00000080:00200000:1.0:1789760783.943225:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760783.943234:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760783.943237:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72744960, 72749056) 0 0 00000080:00200000:1.0:1789760783.943241:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760783.943271:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17760, 17760] 00020000:00200000:1.0:1789760783.943281:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72744960 chunk: [72744960, 72749056) 72749056, 4096 00020000:00200000:1.0:1789760783.943286:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760783.943307:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17760, 17760] 00000008:00010000:1.0:1789760783.943325:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760783.943331:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760783.943345:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72744960, 72749056) append 0 append lockpos 0 00000080:00200000:1.0:1789760783.943361:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72744960=0x4560000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760783.943610:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760783.943614:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72749056 00000020:00010000:1.0:1789760783.943625:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760783.943629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.943641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760783.943682:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760783.943685:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760783.943694:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760783.943700:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760783.943720:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760783.943842:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 72744960<>72749056 00000100:00100000:1.0:1789760783.943881:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760783.943902:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565503744:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.957357:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985068 gen 3 00000008:00010000:0.0:1789760783.957456:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760783.957465:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.957487:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.957514:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760783.957549:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565503744:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.957734:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17760, 17760] 00000080:00200000:2.0:1789760783.957751:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.957758:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.957774:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72749056 i_blocks 142112 00000020:00200000:2.0:1789760783.957778:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.957788:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.957824:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72749056, count: 4096 00000080:00200000:2.0:1789760783.957832:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72749056, bytes: 4096 00000080:00200000:2.0:1789760783.957842:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.957850:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.957852:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72749056, 72753152) 0 0 00000080:00200000:2.0:1789760783.957857:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.957874:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17761, 17761] 00020000:00200000:2.0:1789760783.957883:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72749056 chunk: [72749056, 72753152) 72753152, 4096 00020000:00200000:2.0:1789760783.957887:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.957906:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17761, 17761] 00000008:00010000:2.0:1789760783.957926:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.957932:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.957944:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72749056, 72753152) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.957959:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72749056=0x4561000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.958301:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.958306:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72753152 00000020:00010000:2.0:1789760783.958315:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.958319:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.958322:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.958326:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.958328:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.958336:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.958342:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.958362:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760783.959168:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 72749056<>72753152 00000100:00100000:2.0:1789760783.959233:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565503872:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.966943:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985069 gen 3 00000008:00010000:2.0:1789760783.966994:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760783.967003:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.967015:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.967086:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760783.967119:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565503872:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.967484:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17761, 17761] 00000080:00200000:0.0:1789760783.967502:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.967508:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.967526:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72753152 i_blocks 142120 00000020:00200000:0.0:1789760783.967530:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.968626:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.968683:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72753152, count: 4096 00000080:00200000:0.0:1789760783.968694:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72753152, bytes: 4096 00000080:00200000:0.0:1789760783.968707:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.968718:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.968721:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72753152, 72757248) 0 0 00000080:00200000:0.0:1789760783.968727:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.968745:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17762, 17762] 00020000:00200000:0.0:1789760783.968756:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72753152 chunk: [72753152, 72757248) 72757248, 4096 00020000:00200000:0.0:1789760783.968760:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.968782:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17762, 17762] 00000008:00010000:0.0:1789760783.968799:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.968806:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.968818:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72753152, 72757248) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.968831:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72753152=0x4562000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.968890:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.968893:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72757248 00000020:00010000:0.0:1789760783.968900:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.968904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.968907:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.968910:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.968913:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.968922:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.968929:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760783.968950:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760783.969078:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72753152<>72757248 00000100:00100000:0.0:1789760783.969135:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760783.969160:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565504000:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760783.981395:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985070 gen 3 00000008:00010000:0.0:1789760783.981460:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760783.981479:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760783.981491:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760783.981583:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760783.981615:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565504000:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760783.981861:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17762, 17762] 00000080:00200000:2.0:1789760783.981879:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760783.981885:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760783.981900:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72757248 i_blocks 142128 00000020:00200000:2.0:1789760783.981904:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760783.981916:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760783.981953:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72757248, count: 4096 00000080:00200000:2.0:1789760783.981960:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72757248, bytes: 4096 00000080:00200000:2.0:1789760783.981971:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760783.981990:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760783.981992:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72757248, 72761344) 0 0 00000080:00200000:2.0:1789760783.981997:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760783.982014:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17763, 17763] 00020000:00200000:2.0:1789760783.982024:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72757248 chunk: [72757248, 72761344) 72761344, 4096 00020000:00200000:2.0:1789760783.982029:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760783.982051:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17763, 17763] 00000008:00010000:2.0:1789760783.982071:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760783.982077:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760783.982092:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72757248, 72761344) append 0 append lockpos 0 00000080:00200000:2.0:1789760783.982106:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72757248=0x4563000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760783.982500:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760783.982504:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72761344 00000020:00010000:2.0:1789760783.982513:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760783.982568:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.982572:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760783.982577:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760783.982579:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760783.982589:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760783.982598:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760783.982623:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760783.982908:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 72757248<>72761344 00000100:00100000:3.0:1789760783.982989:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565504128:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760783.998877:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985071 gen 3 00000008:00010000:2.0:1789760783.998920:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760783.998927:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760783.998936:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760783.998953:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000080:00200000:0.0:1789760783.998986:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17763, 17763] 00000100:00100000:2.0:1789760783.998992:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565504128:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760783.999003:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760783.999009:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760783.999025:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72761344 i_blocks 142136 00000020:00200000:0.0:1789760783.999029:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760783.999039:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760783.999076:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72761344, count: 4096 00000080:00200000:0.0:1789760783.999085:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72761344, bytes: 4096 00000080:00200000:0.0:1789760783.999094:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760783.999103:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760783.999106:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72761344, 72765440) 0 0 00000080:00200000:0.0:1789760783.999111:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760783.999128:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17764, 17764] 00020000:00200000:0.0:1789760783.999138:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72761344 chunk: [72761344, 72765440) 72765440, 4096 00020000:00200000:0.0:1789760783.999142:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760783.999162:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17764, 17764] 00000008:00010000:0.0:1789760783.999179:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760783.999186:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760783.999197:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72761344, 72765440) append 0 append lockpos 0 00000080:00200000:0.0:1789760783.999212:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72761344=0x4564000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760783.999270:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760783.999273:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72765440 00000020:00010000:0.0:1789760783.999282:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760783.999286:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.999495:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760783.999499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760783.999502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760783.999512:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760783.999520:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760783.999538:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760783.999690:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72761344<>72765440 00000100:00100000:1.0:1789760783.999768:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565504256:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.012400:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985072 gen 3 00000008:00010000:0.0:1789760784.012603:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.012610:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.012618:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.012721:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.012745:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565504256:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.012896:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17764, 17764] 00000080:00200000:2.0:1789760784.012908:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.012913:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.012925:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72765440 i_blocks 142144 00000020:00200000:2.0:1789760784.012928:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.012938:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.012961:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72765440, count: 4096 00000080:00200000:2.0:1789760784.012967:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72765440, bytes: 4096 00000080:00200000:2.0:1789760784.012975:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.012982:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.012984:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72765440, 72769536) 0 0 00000080:00200000:2.0:1789760784.012987:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.012999:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17765, 17765] 00020000:00200000:2.0:1789760784.013006:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72765440 chunk: [72765440, 72769536) 72769536, 4096 00020000:00200000:2.0:1789760784.013010:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.013024:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17765, 17765] 00000008:00010000:2.0:1789760784.013039:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.013044:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.013052:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72765440, 72769536) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.013064:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72765440=0x4565000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.013106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.013110:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72769536 00000020:00010000:2.0:1789760784.013116:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.013119:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.013123:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.013126:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.013128:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.013135:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.013140:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.013153:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760784.013245:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 72765440<>72769536 00000100:00100000:3.0:1789760784.013312:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565504384:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.028822:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985073 gen 3 00000008:00010000:3.0:1789760784.028857:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760784.028863:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.028871:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.028892:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760784.028913:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565504384:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.029421:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17765, 17765] 00000080:00200000:2.0:1789760784.029440:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.029445:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.029473:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72769536 i_blocks 142152 00000020:00200000:2.0:1789760784.029477:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.029489:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.029521:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72769536, count: 4096 00000080:00200000:2.0:1789760784.029531:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72769536, bytes: 4096 00000080:00200000:2.0:1789760784.029542:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.029550:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.029553:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72769536, 72773632) 0 0 00000080:00200000:2.0:1789760784.029557:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.029569:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17766, 17766] 00020000:00200000:2.0:1789760784.029576:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72769536 chunk: [72769536, 72773632) 72773632, 4096 00020000:00200000:2.0:1789760784.029579:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.029595:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17766, 17766] 00000008:00010000:2.0:1789760784.029616:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.029621:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.029636:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72769536, 72773632) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.029674:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72769536=0x4566000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.029738:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.029744:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72773632 00000020:00010000:2.0:1789760784.029751:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.029754:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.029757:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.029759:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.029764:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.029771:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.029778:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.029797:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f8441893c0 00000008:00100000:2.0:1789760784.029919:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 72769536<>72773632 00000100:00100000:3.0:1789760784.030117:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565504512:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.043353:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985074 gen 3 00000008:00010000:3.0:1789760784.043402:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760784.043412:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.043422:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.043442:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760784.043481:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565504512:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.045167:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17766, 17766] 00000080:00200000:2.0:1789760784.045187:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.045192:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.045205:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72773632 i_blocks 142160 00000020:00200000:2.0:1789760784.045210:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.045222:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.045246:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72773632, count: 4096 00000080:00200000:2.0:1789760784.045254:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72773632, bytes: 4096 00000080:00200000:2.0:1789760784.045263:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.045271:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.045273:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72773632, 72777728) 0 0 00000080:00200000:2.0:1789760784.045276:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.045290:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17767, 17767] 00020000:00200000:2.0:1789760784.045297:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72773632 chunk: [72773632, 72777728) 72777728, 4096 00020000:00200000:2.0:1789760784.045301:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.045321:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17767, 17767] 00000008:00010000:2.0:1789760784.045336:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.045341:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.045382:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72773632, 72777728) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.045394:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72773632=0x4567000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.045445:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.045448:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72777728 00000020:00010000:2.0:1789760784.045473:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.045477:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.045480:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.045484:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.045486:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.045493:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.045499:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.045518:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.045703:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 72773632<>72777728 00000100:00100000:3.0:1789760784.045794:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565504640:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.059053:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985075 gen 3 00000008:00010000:3.0:1789760784.059103:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760784.059111:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.059121:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.059200:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760784.059231:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565504640:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.059316:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17767, 17767] 00000080:00200000:0.0:1789760784.059332:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.059576:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.059593:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72777728 i_blocks 142168 00000020:00200000:0.0:1789760784.059596:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.059606:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.059641:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72777728, count: 4096 00000080:00200000:0.0:1789760784.059672:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72777728, bytes: 4096 00000080:00200000:0.0:1789760784.059683:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.059691:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.059693:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72777728, 72781824) 0 0 00000080:00200000:0.0:1789760784.059698:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.059716:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17768, 17768] 00020000:00200000:0.0:1789760784.059808:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72777728 chunk: [72777728, 72781824) 72781824, 4096 00020000:00200000:0.0:1789760784.059873:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.059895:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17768, 17768] 00000008:00010000:0.0:1789760784.059914:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.059920:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.059932:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72777728, 72781824) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.060009:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72777728=0x4568000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.060075:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.060078:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72781824 00000020:00010000:0.0:1789760784.060087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.060090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.060093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.060096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.060099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.060107:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.060113:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.060134:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.060253:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72777728<>72781824 00000100:00100000:1.0:1789760784.060351:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565504768:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.079750:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985076 gen 3 00000008:00010000:0.0:1789760784.079800:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.079808:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.079818:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.079843:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.079871:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565504768:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.079921:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17768, 17768] 00000080:00200000:2.0:1789760784.079941:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.079947:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.079960:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72781824 i_blocks 142176 00000020:00200000:2.0:1789760784.079965:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.079978:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.080002:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72781824, count: 4096 00000080:00200000:2.0:1789760784.080010:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72781824, bytes: 4096 00000080:00200000:2.0:1789760784.080021:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.080028:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.080030:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72781824, 72785920) 0 0 00000080:00200000:2.0:1789760784.080034:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.080047:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17769, 17769] 00020000:00200000:2.0:1789760784.080054:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72781824 chunk: [72781824, 72785920) 72785920, 4096 00020000:00200000:2.0:1789760784.080057:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.080073:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17769, 17769] 00000008:00010000:2.0:1789760784.080087:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.080091:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.080101:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72781824, 72785920) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.080112:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72781824=0x4569000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.080176:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.080179:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72785920 00000020:00010000:2.0:1789760784.080185:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.080188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.080191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.080194:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.080196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.080203:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.080208:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.080227:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.080327:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 72781824<>72785920 00000100:00100000:3.0:1789760784.080527:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565504896:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.094640:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985077 gen 3 00000008:00010000:2.0:1789760784.095582:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760784.095593:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.095604:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.095629:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760784.095690:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565504896:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.096006:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17769, 17769] 00000080:00200000:0.0:1789760784.096022:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.096028:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.096042:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72785920 i_blocks 142184 00000020:00200000:0.0:1789760784.096045:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.096055:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.096089:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72785920, count: 4096 00000080:00200000:0.0:1789760784.096097:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72785920, bytes: 4096 00000080:00200000:0.0:1789760784.096110:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.096119:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.096122:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72785920, 72790016) 0 0 00000080:00200000:0.0:1789760784.096126:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.096143:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17770, 17770] 00020000:00200000:0.0:1789760784.096152:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72785920 chunk: [72785920, 72790016) 72790016, 4096 00020000:00200000:0.0:1789760784.096157:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.096178:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17770, 17770] 00000008:00010000:0.0:1789760784.096202:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.096207:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.096220:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72785920, 72790016) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.096235:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72785920=0x456a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.096295:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.096298:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72790016 00000020:00010000:0.0:1789760784.096306:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.096310:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.096313:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.096316:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.096319:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.096328:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.096336:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.096352:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.096501:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72785920<>72790016 00000100:00100000:1.0:1789760784.096583:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565505024:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.108838:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985078 gen 3 00000008:00010000:0.0:1789760784.108892:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.108913:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.108926:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.108953:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.108982:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565505024:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.109004:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17770, 17770] 00000080:00200000:2.0:1789760784.109035:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.109042:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.109058:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72790016 i_blocks 142192 00000020:00200000:2.0:1789760784.109062:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.109074:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.109118:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72790016, count: 4096 00000080:00200000:2.0:1789760784.109126:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72790016, bytes: 4096 00000080:00200000:2.0:1789760784.109138:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.109147:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.109149:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72790016, 72794112) 0 0 00000080:00200000:2.0:1789760784.109154:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.109172:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17771, 17771] 00020000:00200000:2.0:1789760784.109182:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72790016 chunk: [72790016, 72794112) 72794112, 4096 00020000:00200000:2.0:1789760784.109187:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.109205:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17771, 17771] 00000008:00010000:2.0:1789760784.109222:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.109228:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.109240:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72790016, 72794112) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.109254:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72790016=0x456b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.109316:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.109319:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72794112 00000020:00010000:2.0:1789760784.109328:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.109331:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.109335:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.109338:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.109341:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.109349:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.109356:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.109371:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.109517:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 72790016<>72794112 00000100:00100000:2.0:1789760784.109564:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760784.109592:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565505152:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.126217:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985079 gen 3 00000008:00010000:2.0:1789760784.126275:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760784.126285:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.126298:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.126320:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760784.126348:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565505152:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760784.126391:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17771, 17771] 00000080:00200000:3.0:1789760784.126419:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760784.126426:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.127278:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72794112 i_blocks 142200 00000020:00200000:2.0:1789760784.127286:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.127299:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.127333:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72794112, count: 4096 00000080:00200000:2.0:1789760784.127341:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72794112, bytes: 4096 00000080:00200000:2.0:1789760784.127354:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.127363:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.127365:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72794112, 72798208) 0 0 00000080:00200000:2.0:1789760784.127370:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.127388:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17772, 17772] 00020000:00200000:2.0:1789760784.127398:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72794112 chunk: [72794112, 72798208) 72798208, 4096 00020000:00200000:2.0:1789760784.127403:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.128514:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17772, 17772] 00000008:00010000:2.0:1789760784.128546:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.128553:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.128567:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72794112, 72798208) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.128584:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72794112=0x456c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.128674:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.128677:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72798208 00000020:00010000:2.0:1789760784.128685:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.128689:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.128692:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.128696:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.128700:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.128711:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.128719:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.129712:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.129943:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 72794112<>72798208 00000100:00100000:3.0:1789760784.130021:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760784.130323:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565505280:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.141754:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985080 gen 3 00000008:00010000:2.0:1789760784.141803:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760784.141811:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.141879:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.141955:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760784.141997:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565505280:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760784.142191:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17772, 17772] 00000080:00200000:3.0:1789760784.142207:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760784.142213:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.142366:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72798208 i_blocks 142208 00000020:00200000:2.0:1789760784.142372:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.142383:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.142480:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72798208, count: 4096 00000080:00200000:2.0:1789760784.142543:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72798208, bytes: 4096 00000080:00200000:2.0:1789760784.142555:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.142563:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.142566:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72798208, 72802304) 0 0 00000080:00200000:2.0:1789760784.142571:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.142589:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17773, 17773] 00020000:00200000:2.0:1789760784.142598:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72798208 chunk: [72798208, 72802304) 72802304, 4096 00020000:00200000:2.0:1789760784.142604:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.142625:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17773, 17773] 00000008:00010000:2.0:1789760784.142676:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.142682:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.142694:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72798208, 72802304) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.142709:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72798208=0x456d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.142765:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.142769:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72802304 00000020:00010000:2.0:1789760784.142777:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.142781:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.142784:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.142787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.142791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.142799:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.142807:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.142919:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.143743:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 72798208<>72802304 00000100:00100000:3.0:1789760784.143840:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565505408:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.154353:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985081 gen 3 00000008:00010000:3.0:1789760784.154407:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760784.154416:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.154429:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.154454:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000080:00200000:2.0:1789760784.154571:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17773, 17773] 00000080:00200000:2.0:1789760784.154589:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.154595:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.154610:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72802304 i_blocks 142216 00000020:00200000:2.0:1789760784.154614:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.154623:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000100:00100000:3.0:1789760784.154756:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565505408:192.168.203.147@tcp:4:dd.0 00000080:00200008:2.0:1789760784.154852:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72802304, count: 4096 00000080:00200000:2.0:1789760784.154860:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72802304, bytes: 4096 00000080:00200000:2.0:1789760784.154869:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.154878:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.154881:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72802304, 72806400) 0 0 00000080:00200000:2.0:1789760784.154885:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.154903:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17774, 17774] 00020000:00200000:2.0:1789760784.154912:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72802304 chunk: [72802304, 72806400) 72806400, 4096 00020000:00200000:2.0:1789760784.154916:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.154935:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17774, 17774] 00000008:00010000:2.0:1789760784.154952:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.154958:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.154969:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72802304, 72806400) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.154982:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72802304=0x456e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.155092:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.155095:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72806400 00000020:00010000:2.0:1789760784.155104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.155108:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.155111:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.155114:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.155117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.155125:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.155135:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.155150:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.155263:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 72802304<>72806400 00000100:00100000:3.0:1789760784.155331:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565505536:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.174806:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985082 gen 3 00000008:00010000:2.0:1789760784.175382:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760784.175391:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.175402:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.175483:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:2.0:1789760784.175513:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565505536:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.176054:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17774, 17774] 00000080:00200000:0.0:1789760784.176071:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.176077:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.176093:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72806400 i_blocks 142224 00000020:00200000:0.0:1789760784.176097:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.176107:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.176140:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72806400, count: 4096 00000080:00200000:0.0:1789760784.176148:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72806400, bytes: 4096 00000080:00200000:0.0:1789760784.176163:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.176172:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.176175:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72806400, 72810496) 0 0 00000080:00200000:0.0:1789760784.176180:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.176199:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17775, 17775] 00020000:00200000:0.0:1789760784.176209:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72806400 chunk: [72806400, 72810496) 72810496, 4096 00020000:00200000:0.0:1789760784.176215:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.176237:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17775, 17775] 00000008:00010000:0.0:1789760784.176255:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.176261:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.176273:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72806400, 72810496) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.176287:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72806400=0x456f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.176344:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.176347:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72810496 00000020:00010000:0.0:1789760784.176355:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.176359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.176362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.176365:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.176368:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.176375:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.176381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.176401:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.176537:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72806400<>72810496 00000100:00100000:1.0:1789760784.176696:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565505664:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.187520:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985083 gen 3 00000008:00010000:0.0:1789760784.187568:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.187576:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.187587:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.187804:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.187845:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565505664:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.188240:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17775, 17775] 00000080:00200000:2.0:1789760784.188256:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.188261:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.188277:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72810496 i_blocks 142232 00000020:00200000:2.0:1789760784.188280:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.188291:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.188325:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72810496, count: 4096 00000080:00200000:2.0:1789760784.188332:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72810496, bytes: 4096 00000080:00200000:2.0:1789760784.188343:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.188351:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.188353:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72810496, 72814592) 0 0 00000080:00200000:2.0:1789760784.188358:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.188373:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17776, 17776] 00020000:00200000:2.0:1789760784.188381:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72810496 chunk: [72810496, 72814592) 72814592, 4096 00020000:00200000:2.0:1789760784.188386:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.188475:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17776, 17776] 00000008:00010000:2.0:1789760784.188494:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.188499:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.188510:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72810496, 72814592) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.188524:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72810496=0x4570000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.188582:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.188585:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72814592 00000020:00010000:2.0:1789760784.188593:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.188596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.188598:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.188602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.188604:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.188611:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.188617:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.188634:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.188777:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 72810496<>72814592 00000100:00100000:3.0:1789760784.189007:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565505792:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.202934:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985084 gen 3 00000008:00010000:3.0:1789760784.202986:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760784.202994:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.203005:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.203030:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760784.203061:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565505792:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.203171:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17776, 17776] 00000080:00200000:2.0:1789760784.203188:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.203194:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.203211:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72814592 i_blocks 142240 00000020:00200000:2.0:1789760784.203215:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.203225:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.203259:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72814592, count: 4096 00000080:00200000:2.0:1789760784.203266:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72814592, bytes: 4096 00000080:00200000:2.0:1789760784.203275:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.203284:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.203286:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72814592, 72818688) 0 0 00000080:00200000:2.0:1789760784.203291:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.203308:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17777, 17777] 00020000:00200000:2.0:1789760784.203317:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72814592 chunk: [72814592, 72818688) 72818688, 4096 00020000:00200000:2.0:1789760784.203321:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.203341:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17777, 17777] 00000008:00010000:2.0:1789760784.203359:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.203366:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.203378:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72814592, 72818688) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.203392:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72814592=0x4571000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.203462:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.203473:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72818688 00000020:00010000:2.0:1789760784.203482:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.203486:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.203489:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.203493:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.203495:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.203504:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.203511:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.203527:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.203675:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 72814592<>72818688 00000100:00100000:3.0:1789760784.203788:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565505920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.221721:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985085 gen 3 00000008:00010000:3.0:1789760784.221771:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760784.221779:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.221791:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.221812:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760784.221841:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565505920:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.221953:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17777, 17777] 00000080:00200000:2.0:1789760784.221971:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.221977:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.221994:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72818688 i_blocks 142248 00000020:00200000:2.0:1789760784.221997:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.222008:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.222038:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72818688, count: 4096 00000080:00200000:2.0:1789760784.222046:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72818688, bytes: 4096 00000080:00200000:2.0:1789760784.222056:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.222065:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.222067:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72818688, 72822784) 0 0 00000080:00200000:2.0:1789760784.222072:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.222089:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17778, 17778] 00020000:00200000:2.0:1789760784.222097:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72818688 chunk: [72818688, 72822784) 72822784, 4096 00020000:00200000:2.0:1789760784.222102:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.222122:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17778, 17778] 00000008:00010000:2.0:1789760784.222140:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.222146:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.222159:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72818688, 72822784) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.222174:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72818688=0x4572000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.222233:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.222236:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72822784 00000020:00010000:2.0:1789760784.222245:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.222248:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.222252:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.222255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.222258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.222266:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.222274:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.222290:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.222416:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 72818688<>72822784 00000100:00100000:3.0:1789760784.222594:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760784.222613:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565506048:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.238327:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985086 gen 3 00000008:00010000:3.0:1789760784.238381:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760784.238390:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.238401:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.238544:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760784.238577:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565506048:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.238743:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17778, 17778] 00000080:00200000:2.0:1789760784.238761:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.238768:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.238786:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72822784 i_blocks 142256 00000020:00200000:2.0:1789760784.238790:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.238801:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.238836:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72822784, count: 4096 00000080:00200000:2.0:1789760784.238843:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72822784, bytes: 4096 00000080:00200000:2.0:1789760784.238854:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.238863:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.238865:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72822784, 72826880) 0 0 00000080:00200000:2.0:1789760784.238869:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.238885:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17779, 17779] 00020000:00200000:2.0:1789760784.238894:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72822784 chunk: [72822784, 72826880) 72826880, 4096 00020000:00200000:2.0:1789760784.238899:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.238919:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17779, 17779] 00000008:00010000:2.0:1789760784.238936:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.238942:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.238954:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72822784, 72826880) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.238969:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72822784=0x4573000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.239028:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.239031:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72826880 00000020:00010000:2.0:1789760784.239039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.239043:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.239046:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.239049:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.239052:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.239060:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.239067:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.239085:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.239210:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 72822784<>72826880 00000100:00100000:3.0:1789760784.239769:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760784.239792:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565506176:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.251974:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985087 gen 3 00000008:00010000:3.0:1789760784.252022:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760784.252030:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.252042:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.252064:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760784.252093:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565506176:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.252303:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17779, 17779] 00000080:00200000:2.0:1789760784.252323:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.252328:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.252341:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72826880 i_blocks 142264 00000020:00200000:2.0:1789760784.252344:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.252354:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.252386:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72826880, count: 4096 00000080:00200000:2.0:1789760784.252392:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72826880, bytes: 4096 00000080:00200000:2.0:1789760784.252402:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.252410:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.252412:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72826880, 72830976) 0 0 00000080:00200000:2.0:1789760784.252416:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.252431:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17780, 17780] 00020000:00200000:2.0:1789760784.252440:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72826880 chunk: [72826880, 72830976) 72830976, 4096 00020000:00200000:2.0:1789760784.252551:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.252574:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17780, 17780] 00000008:00010000:2.0:1789760784.252592:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.252597:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.252609:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72826880, 72830976) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.252622:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72826880=0x4574000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.252701:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.252703:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72830976 00000020:00010000:2.0:1789760784.252711:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.252715:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.252718:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.252721:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.252724:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.252732:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.252739:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.252757:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.252874:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 72826880<>72830976 00000100:00100000:2.0:1789760784.252920:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760784.252937:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565506304:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.267919:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985088 gen 3 00000008:00010000:2.0:1789760784.267970:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760784.267979:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.267989:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.268014:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:2.0:1789760784.268043:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565506304:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.268160:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17780, 17780] 00000080:00200000:0.0:1789760784.268176:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.268182:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.268196:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72830976 i_blocks 142272 00000020:00200000:0.0:1789760784.268200:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.268210:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.268256:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72830976, count: 4096 00000080:00200000:0.0:1789760784.268263:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72830976, bytes: 4096 00000080:00200000:0.0:1789760784.268273:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.268282:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.268284:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72830976, 72835072) 0 0 00000080:00200000:0.0:1789760784.268288:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.268305:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17781, 17781] 00020000:00200000:0.0:1789760784.268314:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72830976 chunk: [72830976, 72835072) 72835072, 4096 00020000:00200000:0.0:1789760784.268319:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.268337:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17781, 17781] 00000008:00010000:0.0:1789760784.268679:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.268687:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.268701:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72830976, 72835072) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.268717:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72830976=0x4575000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.268782:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.268785:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72835072 00000020:00010000:0.0:1789760784.268793:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.268796:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.268799:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.268802:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.268805:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.268812:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.268819:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.268837:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.268965:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72830976<>72835072 00000100:00100000:1.0:1789760784.269044:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565506432:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.290178:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985089 gen 3 00000008:00010000:0.0:1789760784.290233:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.290242:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.290253:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.290276:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.290307:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565506432:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.290421:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17781, 17781] 00000080:00200000:2.0:1789760784.290438:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.290444:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.290462:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72835072 i_blocks 142280 00000020:00200000:2.0:1789760784.290475:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.290487:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.290524:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72835072, count: 4096 00000080:00200000:2.0:1789760784.290532:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72835072, bytes: 4096 00000080:00200000:2.0:1789760784.290543:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.290551:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.290554:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72835072, 72839168) 0 0 00000080:00200000:2.0:1789760784.290558:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.290576:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17782, 17782] 00020000:00200000:2.0:1789760784.290586:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72835072 chunk: [72835072, 72839168) 72839168, 4096 00020000:00200000:2.0:1789760784.290590:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.290611:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17782, 17782] 00000008:00010000:2.0:1789760784.290629:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.290636:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.290888:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72835072, 72839168) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.290903:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72835072=0x4576000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.290962:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.290966:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72839168 00000020:00010000:2.0:1789760784.290975:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.290979:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.290982:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.290986:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.290989:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.290998:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.291005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.291021:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.291258:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 72835072<>72839168 00000100:00100000:3.0:1789760784.291339:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565506560:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.321321:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985090 gen 3 00000008:00010000:3.0:1789760784.321371:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760784.321380:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.321391:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.321480:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760784.321521:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565506560:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.321827:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17782, 17782] 00000080:00200000:2.0:1789760784.321845:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.321851:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.322070:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72839168 i_blocks 142288 00000020:00200000:2.0:1789760784.322076:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.322088:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.322129:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72839168, count: 4096 00000080:00200000:2.0:1789760784.322137:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72839168, bytes: 4096 00000080:00200000:2.0:1789760784.322149:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.322157:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.322159:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72839168, 72843264) 0 0 00000080:00200000:2.0:1789760784.322164:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.322184:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17783, 17783] 00020000:00200000:2.0:1789760784.322433:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72839168 chunk: [72839168, 72843264) 72843264, 4096 00020000:00200000:2.0:1789760784.322439:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.322461:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17783, 17783] 00000008:00010000:2.0:1789760784.322558:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.322564:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.322577:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72839168, 72843264) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.322592:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72839168=0x4577000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.322832:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.322836:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72843264 00000020:00010000:2.0:1789760784.322845:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.322849:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.322851:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.322855:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.322858:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.322866:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.322874:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.322898:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.323034:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 72839168<>72843264 00000100:00100000:3.0:1789760784.327403:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565506688:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.338108:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985091 gen 3 00000008:00010000:3.0:1789760784.338158:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760784.338167:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.338179:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.338212:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760784.338251:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565506688:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.338406:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17783, 17783] 00000080:00200000:0.0:1789760784.338424:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.338429:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.338446:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72843264 i_blocks 142296 00000020:00200000:0.0:1789760784.338449:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.338460:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.338506:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72843264, count: 4096 00000080:00200000:0.0:1789760784.338515:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72843264, bytes: 4096 00000080:00200000:0.0:1789760784.338525:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.338534:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.338537:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72843264, 72847360) 0 0 00000080:00200000:0.0:1789760784.338541:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.338560:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17784, 17784] 00020000:00200000:0.0:1789760784.338568:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72843264 chunk: [72843264, 72847360) 72847360, 4096 00020000:00200000:0.0:1789760784.338573:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.338592:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17784, 17784] 00000008:00010000:0.0:1789760784.338610:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.338617:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.338629:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72843264, 72847360) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.338675:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72843264=0x4578000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.339059:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.339063:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72847360 00000020:00010000:0.0:1789760784.339072:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.339076:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.339080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.339084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.339086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.339095:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.339102:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.339122:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.339379:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72843264<>72847360 00000100:00100000:1.0:1789760784.339477:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565506816:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.353606:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985092 gen 3 00000008:00010000:0.0:1789760784.353690:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.353698:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.353708:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.353733:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.353758:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565506816:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.353835:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17784, 17784] 00000080:00200000:2.0:1789760784.353853:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.353859:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.353874:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72847360 i_blocks 142304 00000020:00200000:2.0:1789760784.353883:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.353896:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.353929:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72847360, count: 4096 00000080:00200000:2.0:1789760784.353936:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72847360, bytes: 4096 00000080:00200000:2.0:1789760784.353946:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.353955:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.353957:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72847360, 72851456) 0 0 00000080:00200000:2.0:1789760784.353962:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.353980:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17785, 17785] 00020000:00200000:2.0:1789760784.353989:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72847360 chunk: [72847360, 72851456) 72851456, 4096 00020000:00200000:2.0:1789760784.353994:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.354012:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17785, 17785] 00000008:00010000:2.0:1789760784.354029:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.354036:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.354047:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72847360, 72851456) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.354061:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72847360=0x4579000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.354115:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.354118:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72851456 00000020:00010000:2.0:1789760784.354125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.354129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.354132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.354135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.354138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.354145:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.354152:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.354168:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.354290:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 72847360<>72851456 00000100:00100000:3.0:1789760784.354369:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565506944:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.366533:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985093 gen 3 00000008:00010000:2.0:1789760784.366587:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760784.366596:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.366608:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.366636:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000080:00200000:0.0:1789760784.366717:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17785, 17785] 00000100:00100000:2.0:1789760784.366734:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565506944:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.366738:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.366745:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.366761:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72851456 i_blocks 142312 00000020:00200000:0.0:1789760784.366765:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.366776:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.366809:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72851456, count: 4096 00000080:00200000:0.0:1789760784.366818:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72851456, bytes: 4096 00000080:00200000:0.0:1789760784.366829:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.366838:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.366841:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72851456, 72855552) 0 0 00000080:00200000:0.0:1789760784.366846:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.366864:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17786, 17786] 00020000:00200000:0.0:1789760784.366872:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72851456 chunk: [72851456, 72855552) 72855552, 4096 00020000:00200000:0.0:1789760784.366877:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.366897:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17786, 17786] 00000008:00010000:0.0:1789760784.366915:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.366933:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.366947:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72851456, 72855552) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.366960:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72851456=0x457a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.367020:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.367023:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72855552 00000020:00010000:0.0:1789760784.367032:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.367036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.367039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.367044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.367047:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.367055:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.367062:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.367079:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.368000:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72851456<>72855552 00000100:00100000:1.0:1789760784.368061:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760784.368151:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565507072:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.378530:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985094 gen 3 00000008:00010000:0.0:1789760784.378582:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.378590:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.378600:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.378621:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.378693:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565507072:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760784.378822:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17786, 17786] 00000080:00200000:1.0:1789760784.378844:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760784.378850:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760784.378865:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72855552 i_blocks 142320 00000020:00200000:1.0:1789760784.378869:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760784.378879:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760784.378912:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72855552, count: 4096 00000080:00200000:1.0:1789760784.378920:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72855552, bytes: 4096 00000080:00200000:1.0:1789760784.378930:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760784.378940:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760784.378942:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72855552, 72859648) 0 0 00000080:00200000:1.0:1789760784.378947:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760784.378964:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17787, 17787] 00020000:00200000:1.0:1789760784.378972:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72855552 chunk: [72855552, 72859648) 72859648, 4096 00020000:00200000:1.0:1789760784.378977:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760784.378996:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17787, 17787] 00000008:00010000:1.0:1789760784.379013:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760784.379019:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760784.379030:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72855552, 72859648) append 0 append lockpos 0 00000080:00200000:1.0:1789760784.379045:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72855552=0x457b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760784.379096:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760784.379099:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72859648 00000020:00010000:1.0:1789760784.379107:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760784.379110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.379114:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.379118:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760784.379120:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760784.379128:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760784.379135:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.380051:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:0.0:1789760784.380367:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72855552<>72859648 00000100:00100000:1.0:1789760784.380536:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760784.380641:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565507200:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.392324:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985095 gen 3 00000008:00010000:0.0:1789760784.392377:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760784.392385:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.392396:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.392524:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760784.392555:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565507200:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760784.394292:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17787, 17787] 00000080:00200000:1.0:1789760784.394318:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760784.394325:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760784.394344:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72859648 i_blocks 142328 00000020:00200000:1.0:1789760784.394349:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760784.394376:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760784.394411:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72859648, count: 4096 00000080:00200000:1.0:1789760784.394419:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72859648, bytes: 4096 00000080:00200000:1.0:1789760784.394435:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760784.394445:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760784.394447:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72859648, 72863744) 0 0 00000080:00200000:1.0:1789760784.394452:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760784.394484:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17788, 17788] 00020000:00200000:1.0:1789760784.394493:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72859648 chunk: [72859648, 72863744) 72863744, 4096 00020000:00200000:1.0:1789760784.394497:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760784.394520:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17788, 17788] 00000008:00010000:1.0:1789760784.394539:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760784.394546:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760784.394558:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72859648, 72863744) append 0 append lockpos 0 00000080:00200000:1.0:1789760784.394574:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72859648=0x457c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760784.394683:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760784.394687:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72863744 00000020:00010000:1.0:1789760784.394695:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760784.394699:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.394702:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.394706:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760784.394709:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760784.394717:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760784.394724:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760784.394751:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760784.394882:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 72859648<>72863744 00000100:00100000:1.0:1789760784.394935:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760784.394963:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565507328:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.406003:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985096 gen 3 00000008:00010000:0.0:1789760784.406052:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760784.406061:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.406071:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.406563:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760784.406760:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565507328:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.407573:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17788, 17788] 00000080:00200000:2.0:1789760784.407594:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.407601:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.407617:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72863744 i_blocks 142336 00000020:00200000:2.0:1789760784.407621:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.407631:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.407752:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72863744, count: 4096 00000080:00200000:2.0:1789760784.407761:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72863744, bytes: 4096 00000080:00200000:2.0:1789760784.407774:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.407784:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.407786:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72863744, 72867840) 0 0 00000080:00200000:2.0:1789760784.407791:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.407809:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17789, 17789] 00020000:00200000:2.0:1789760784.407818:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72863744 chunk: [72863744, 72867840) 72867840, 4096 00020000:00200000:2.0:1789760784.407823:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.407845:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17789, 17789] 00000008:00010000:2.0:1789760784.407864:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.407870:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.407883:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72863744, 72867840) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.407897:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72863744=0x457d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.407954:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.407956:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72867840 00000020:00010000:2.0:1789760784.407964:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.407967:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.407970:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.407974:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.407977:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.407984:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.407991:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.408009:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.408130:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 72863744<>72867840 00000100:00100000:2.0:1789760784.408173:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760784.408191:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565507456:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.423361:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985097 gen 3 00000008:00010000:2.0:1789760784.423413:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760784.423422:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.423434:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.423530:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760784.423561:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565507456:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760784.426918:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17789, 17789] 00000080:00200000:3.0:1789760784.426984:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760784.426992:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.432399:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72867840 i_blocks 142344 00000020:00200000:2.0:1789760784.432414:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.432435:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.432556:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72867840, count: 4096 00000080:00200000:2.0:1789760784.432567:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72867840, bytes: 4096 00000080:00200000:2.0:1789760784.432584:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.432593:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.432595:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72867840, 72871936) 0 0 00000080:00200000:2.0:1789760784.432600:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.432619:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17790, 17790] 00020000:00200000:2.0:1789760784.432629:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72867840 chunk: [72867840, 72871936) 72871936, 4096 00020000:00200000:2.0:1789760784.432635:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.433407:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17790, 17790] 00000008:00010000:2.0:1789760784.433432:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.433438:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.433528:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72867840, 72871936) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.433546:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72867840=0x457e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.433623:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.433627:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72871936 00000020:00010000:2.0:1789760784.433636:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.433641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.433735:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.433740:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.433743:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.433754:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.433761:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.433825:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.433950:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 72867840<>72871936 00000100:00100000:3.0:1789760784.434047:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565507584:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.451194:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985098 gen 3 00000008:00010000:2.0:1789760784.451333:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760784.451343:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.451665:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.451749:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760784.451780:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565507584:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.454670:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17790, 17790] 00000080:00200000:0.0:1789760784.454698:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.454705:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.454723:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72871936 i_blocks 142352 00000020:00200000:0.0:1789760784.454728:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.454746:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.454783:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72871936, count: 4096 00000080:00200000:0.0:1789760784.454791:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72871936, bytes: 4096 00000080:00200000:0.0:1789760784.454806:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.454814:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.454817:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72871936, 72876032) 0 0 00000080:00200000:0.0:1789760784.454821:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.454947:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17791, 17791] 00020000:00200000:0.0:1789760784.454957:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72871936 chunk: [72871936, 72876032) 72876032, 4096 00020000:00200000:0.0:1789760784.454962:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.455045:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17791, 17791] 00000008:00010000:0.0:1789760784.455065:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.455071:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.455084:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72871936, 72876032) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.455098:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72871936=0x457f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.455168:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.455172:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72876032 00000020:00010000:0.0:1789760784.455181:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.455185:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.455188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.455192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.455195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.455204:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.455211:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.455321:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.455517:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72871936<>72876032 00000100:00100000:1.0:1789760784.455633:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565507712:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.470182:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985099 gen 3 00000008:00010000:0.0:1789760784.470233:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.470243:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.470255:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.470344:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.470387:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565507712:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.471976:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17791, 17791] 00000080:00200000:2.0:1789760784.472074:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.472080:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.472099:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72876032 i_blocks 142360 00000020:00200000:2.0:1789760784.472103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.472121:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.472160:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72876032, count: 4096 00000080:00200000:2.0:1789760784.472168:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72876032, bytes: 4096 00000080:00200000:2.0:1789760784.472183:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.472191:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.472193:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72876032, 72880128) 0 0 00000080:00200000:2.0:1789760784.472198:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.472214:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17792, 17792] 00020000:00200000:2.0:1789760784.472222:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72876032 chunk: [72876032, 72880128) 72880128, 4096 00020000:00200000:2.0:1789760784.472227:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.472248:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17792, 17792] 00000008:00010000:2.0:1789760784.472270:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.472278:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.472292:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72876032, 72880128) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.472306:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72876032=0x4580000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.472910:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.472916:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72880128 00000020:00010000:2.0:1789760784.472926:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.472930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.472933:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.472936:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.472939:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.472949:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.472956:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.473458:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.473622:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 72876032<>72880128 00000100:00100000:3.0:1789760784.473900:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565507840:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.487038:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985100 gen 3 00000008:00010000:3.0:1789760784.487091:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760784.487102:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.487114:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.487140:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760784.487170:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565507840:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.487594:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17792, 17792] 00000080:00200000:2.0:1789760784.487611:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.487617:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.487635:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72880128 i_blocks 142368 00000020:00200000:2.0:1789760784.487639:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.487695:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.487733:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72880128, count: 4096 00000080:00200000:2.0:1789760784.487742:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72880128, bytes: 4096 00000080:00200000:2.0:1789760784.487754:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.487763:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.487766:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72880128, 72884224) 0 0 00000080:00200000:2.0:1789760784.487771:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.487788:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17793, 17793] 00020000:00200000:2.0:1789760784.487797:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72880128 chunk: [72880128, 72884224) 72884224, 4096 00020000:00200000:2.0:1789760784.487802:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.487949:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17793, 17793] 00000008:00010000:2.0:1789760784.487967:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.487973:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.487986:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72880128, 72884224) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.488000:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72880128=0x4581000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.488066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.488069:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72884224 00000020:00010000:2.0:1789760784.488078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.488081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.488084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.488087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.488090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.488098:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.488105:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.488126:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.488257:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 72880128<>72884224 00000100:00100000:3.0:1789760784.488527:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760784.489285:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565507968:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.499338:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985101 gen 3 00000008:00010000:3.0:1789760784.499390:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760784.499397:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.499420:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.499443:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760784.499481:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565507968:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.499772:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17793, 17793] 00000080:00200000:2.0:1789760784.499789:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.499795:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.499819:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72884224 i_blocks 142376 00000020:00200000:2.0:1789760784.499823:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.499833:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.499868:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72884224, count: 4096 00000080:00200000:2.0:1789760784.499876:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72884224, bytes: 4096 00000080:00200000:2.0:1789760784.499887:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.499896:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.499898:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72884224, 72888320) 0 0 00000080:00200000:2.0:1789760784.499903:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.499920:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17794, 17794] 00020000:00200000:2.0:1789760784.499930:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72884224 chunk: [72884224, 72888320) 72888320, 4096 00020000:00200000:2.0:1789760784.499934:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.499954:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17794, 17794] 00000008:00010000:2.0:1789760784.499971:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.499978:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.499989:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72884224, 72888320) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.500003:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72884224=0x4582000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.500062:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.500065:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72888320 00000020:00010000:2.0:1789760784.500073:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.500077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.500080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.500083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.500087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.500095:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.500103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.500121:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.500240:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 72884224<>72888320 00000100:00100000:3.0:1789760784.501082:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508096:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.515016:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985102 gen 3 00000008:00010000:3.0:1789760784.515068:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760784.515077:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.515087:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.515111:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760784.515141:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508096:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.515366:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17794, 17794] 00000080:00200000:2.0:1789760784.515385:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.515392:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.515419:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72888320 i_blocks 142384 00000020:00200000:2.0:1789760784.515423:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.515433:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.515476:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72888320, count: 4096 00000080:00200000:2.0:1789760784.515492:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72888320, bytes: 4096 00000080:00200000:2.0:1789760784.515503:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.515512:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.515515:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72888320, 72892416) 0 0 00000080:00200000:2.0:1789760784.515519:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.515535:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17795, 17795] 00020000:00200000:2.0:1789760784.515544:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72888320 chunk: [72888320, 72892416) 72892416, 4096 00020000:00200000:2.0:1789760784.515549:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.515735:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17795, 17795] 00000008:00010000:2.0:1789760784.515754:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.515761:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.515774:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72888320, 72892416) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.515788:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72888320=0x4583000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.515849:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.515852:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72892416 00000020:00010000:2.0:1789760784.515860:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.515864:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.515868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.515872:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.515875:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.515883:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.515890:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.515908:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.516214:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 72888320<>72892416 00000100:00100000:3.0:1789760784.517199:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760784.517228:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508224:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.529580:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985103 gen 3 00000008:00010000:3.0:1789760784.529634:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760784.529692:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.529703:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.529727:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760784.529760:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508224:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.530131:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17795, 17795] 00000080:00200000:2.0:1789760784.530148:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.530155:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.530170:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72892416 i_blocks 142392 00000020:00200000:2.0:1789760784.530175:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.530185:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.530219:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72892416, count: 4096 00000080:00200000:2.0:1789760784.530227:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72892416, bytes: 4096 00000080:00200000:2.0:1789760784.530238:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.530247:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.530249:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72892416, 72896512) 0 0 00000080:00200000:2.0:1789760784.530253:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.530272:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17796, 17796] 00020000:00200000:2.0:1789760784.530281:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72892416 chunk: [72892416, 72896512) 72896512, 4096 00020000:00200000:2.0:1789760784.530286:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.530306:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17796, 17796] 00000008:00010000:2.0:1789760784.530326:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.530333:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.530345:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72892416, 72896512) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.530361:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72892416=0x4584000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.530496:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.530499:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72896512 00000020:00010000:2.0:1789760784.530509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.530513:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.530516:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.530520:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.530523:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.530530:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.530537:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.530619:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.530768:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 72892416<>72896512 00000100:00100000:3.0:1789760784.531335:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508352:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.556238:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985104 gen 3 00000008:00010000:3.0:1789760784.556288:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760784.556296:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.556308:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.556334:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760784.556364:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508352:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.556674:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17796, 17796] 00000080:00200000:2.0:1789760784.556691:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.556697:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.556714:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72896512 i_blocks 142400 00000020:00200000:2.0:1789760784.556717:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.556728:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.556763:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72896512, count: 4096 00000080:00200000:2.0:1789760784.556771:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72896512, bytes: 4096 00000080:00200000:2.0:1789760784.556783:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.556792:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.556795:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72896512, 72900608) 0 0 00000080:00200000:2.0:1789760784.556800:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.557027:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17797, 17797] 00020000:00200000:2.0:1789760784.557037:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72896512 chunk: [72896512, 72900608) 72900608, 4096 00020000:00200000:2.0:1789760784.557042:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.557064:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17797, 17797] 00000008:00010000:2.0:1789760784.557083:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.557090:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.557101:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72896512, 72900608) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.557114:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72896512=0x4585000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.557172:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.557174:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72900608 00000020:00010000:2.0:1789760784.557183:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.557187:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.557190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.557193:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.557196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.557204:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.557210:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.557227:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.557356:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72896512<>72900608 00000100:00100000:3.0:1789760784.557451:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565508480:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.570040:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985105 gen 3 00000008:00010000:3.0:1789760784.570094:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760784.570104:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.570116:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.570142:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760784.570174:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565508480:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.570296:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17797, 17797] 00000080:00200000:2.0:1789760784.570314:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.570321:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.570337:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72900608 i_blocks 142408 00000020:00200000:2.0:1789760784.570342:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.570352:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.570391:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72900608, count: 4096 00000080:00200000:2.0:1789760784.570399:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72900608, bytes: 4096 00000080:00200000:2.0:1789760784.570409:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.570419:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.570421:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72900608, 72904704) 0 0 00000080:00200000:2.0:1789760784.570427:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.570444:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17798, 17798] 00020000:00200000:2.0:1789760784.570453:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72900608 chunk: [72900608, 72904704) 72904704, 4096 00020000:00200000:2.0:1789760784.570458:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.570489:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17798, 17798] 00000008:00010000:2.0:1789760784.570509:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.570517:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.570529:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72900608, 72904704) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.570543:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72900608=0x4586000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.570606:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.570609:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72904704 00000020:00010000:2.0:1789760784.570617:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.570621:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.570624:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.570627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.570630:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.570638:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.570876:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.570895:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.571025:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 72900608<>72904704 00000100:00100000:3.0:1789760784.572886:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508608:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.587235:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985106 gen 3 00000008:00010000:3.0:1789760784.587290:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760784.587300:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.587313:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.587341:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760784.587373:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508608:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.594384:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17798, 17798] 00000080:00200000:2.0:1789760784.594412:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.594418:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.594435:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72904704 i_blocks 142416 00000020:00200000:2.0:1789760784.594441:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.594457:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.594516:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72904704, count: 4096 00000080:00200000:2.0:1789760784.594525:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72904704, bytes: 4096 00000080:00200000:2.0:1789760784.594541:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.594550:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.594553:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72904704, 72908800) 0 0 00000080:00200000:2.0:1789760784.594557:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.594575:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17799, 17799] 00020000:00200000:2.0:1789760784.594585:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72904704 chunk: [72904704, 72908800) 72908800, 4096 00020000:00200000:2.0:1789760784.594589:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.594613:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17799, 17799] 00000008:00010000:2.0:1789760784.594632:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.594639:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.594720:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72904704, 72908800) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.594735:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72904704=0x4587000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.594804:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.594807:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72908800 00000020:00010000:2.0:1789760784.594815:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.594819:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.594822:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.594824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.594827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.594835:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.594842:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.594864:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.594999:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 72904704<>72908800 00000100:00100000:3.0:1789760784.595145:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760784.595172:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508736:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.616011:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985107 gen 3 00000008:00010000:3.0:1789760784.616064:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760784.616073:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.616084:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.616145:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760784.616175:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508736:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.616365:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17799, 17799] 00000080:00200000:2.0:1789760784.616379:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.616383:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.616395:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72908800 i_blocks 142424 00000020:00200000:2.0:1789760784.616398:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.616457:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.616498:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72908800, count: 4096 00000080:00200000:2.0:1789760784.616506:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72908800, bytes: 4096 00000080:00200000:2.0:1789760784.616515:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.616523:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.616526:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72908800, 72912896) 0 0 00000080:00200000:2.0:1789760784.616531:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.616548:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17800, 17800] 00020000:00200000:2.0:1789760784.616557:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72908800 chunk: [72908800, 72912896) 72912896, 4096 00020000:00200000:2.0:1789760784.616562:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.616582:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17800, 17800] 00000008:00010000:2.0:1789760784.616599:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.616606:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.616618:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72908800, 72912896) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.616632:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72908800=0x4588000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.616849:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.616852:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72912896 00000020:00010000:2.0:1789760784.616861:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.616864:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.616868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.616872:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.616875:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.616883:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.616890:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.616901:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.617017:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 72908800<>72912896 00000100:00100000:3.0:1789760784.617123:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508864:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.626363:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985108 gen 3 00000008:00010000:3.0:1789760784.626416:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760784.626424:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.626434:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.626456:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760784.626500:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508864:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.627113:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17800, 17800] 00000080:00200000:2.0:1789760784.627131:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.627136:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.627152:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72912896 i_blocks 142432 00000020:00200000:2.0:1789760784.627155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.627166:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.627198:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72912896, count: 4096 00000080:00200000:2.0:1789760784.627205:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72912896, bytes: 4096 00000080:00200000:2.0:1789760784.627227:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.627236:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.627238:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72912896, 72916992) 0 0 00000080:00200000:2.0:1789760784.627242:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.627258:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17801, 17801] 00020000:00200000:2.0:1789760784.627267:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72912896 chunk: [72912896, 72916992) 72916992, 4096 00020000:00200000:2.0:1789760784.627272:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.627290:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17801, 17801] 00000008:00010000:2.0:1789760784.627308:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.627315:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.627327:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72912896, 72916992) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.627341:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72912896=0x4589000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.627398:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.627414:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72916992 00000020:00010000:2.0:1789760784.627422:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.627426:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.627429:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.627433:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.627436:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.627444:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.627450:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.627478:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760784.627613:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 72912896<>72916992 00000100:00100000:3.0:1789760784.627903:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760784.627926:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508992:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.639512:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985109 gen 3 00000008:00010000:3.0:1789760784.639571:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760784.639579:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.639590:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.639613:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760784.639682:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565508992:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.640304:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17801, 17801] 00000080:00200000:2.0:1789760784.640322:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.640329:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.640346:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72916992 i_blocks 142440 00000020:00200000:2.0:1789760784.640350:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.640362:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.640397:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72916992, count: 4096 00000080:00200000:2.0:1789760784.640795:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72916992, bytes: 4096 00000080:00200000:2.0:1789760784.640880:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.640890:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.640892:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72916992, 72921088) 0 0 00000080:00200000:2.0:1789760784.640896:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.640914:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17802, 17802] 00020000:00200000:2.0:1789760784.640923:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72916992 chunk: [72916992, 72921088) 72921088, 4096 00020000:00200000:2.0:1789760784.640927:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.640950:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17802, 17802] 00000008:00010000:2.0:1789760784.640967:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.640974:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.640986:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72916992, 72921088) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.640999:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72916992=0x458a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.641054:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.641057:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72921088 00000020:00010000:2.0:1789760784.641065:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.641068:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.641071:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.641074:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.641077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.641085:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.641092:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760784.642061:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:3.0:1789760784.642200:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 72916992<>72921088 00000100:00100000:3.0:1789760784.642273:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565509120:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.657988:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985110 gen 3 00000008:00010000:3.0:1789760784.658048:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760784.658103:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.658114:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.658140:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760784.658171:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565509120:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.659052:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17802, 17802] 00000080:00200000:2.0:1789760784.659074:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.659080:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.659098:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72921088 i_blocks 142448 00000020:00200000:2.0:1789760784.659103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.659116:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.659151:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72921088, count: 4096 00000080:00200000:2.0:1789760784.659159:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72921088, bytes: 4096 00000080:00200000:2.0:1789760784.659171:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.659179:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.659182:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72921088, 72925184) 0 0 00000080:00200000:2.0:1789760784.659187:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.659204:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17803, 17803] 00020000:00200000:2.0:1789760784.659225:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72921088 chunk: [72921088, 72925184) 72925184, 4096 00020000:00200000:2.0:1789760784.659229:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.659254:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17803, 17803] 00000008:00010000:2.0:1789760784.659275:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.659281:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.659295:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72921088, 72925184) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.659309:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72921088=0x458b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.659370:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.659373:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72925184 00000020:00010000:2.0:1789760784.659381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.659385:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.659388:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.659392:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.659394:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.659403:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.659410:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.659429:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.659938:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72921088<>72925184 00000100:00100000:2.0:1789760784.659981:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760784.660003:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565509248:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.672858:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985111 gen 3 00000008:00010000:2.0:1789760784.672911:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760784.672920:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.672930:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.672953:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760784.672982:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565509248:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760784.673155:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17803, 17803] 00000080:00200000:3.0:1789760784.673204:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760784.673211:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760784.673227:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72925184 i_blocks 142456 00000020:00200000:3.0:1789760784.673231:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760784.673300:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760784.673352:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72925184, count: 4096 00000080:00200000:3.0:1789760784.673362:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72925184, bytes: 4096 00000080:00200000:3.0:1789760784.673375:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760784.673384:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760784.673386:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72925184, 72929280) 0 0 00000080:00200000:3.0:1789760784.673391:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760784.673409:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17804, 17804] 00020000:00200000:3.0:1789760784.673418:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72925184 chunk: [72925184, 72929280) 72929280, 4096 00020000:00200000:3.0:1789760784.673422:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760784.673441:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17804, 17804] 00000008:00010000:3.0:1789760784.673457:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760784.673473:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760784.673485:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72925184, 72929280) append 0 append lockpos 0 00000080:00200000:3.0:1789760784.673498:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72925184=0x458c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760784.673771:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760784.673774:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72929280 00000020:00010000:3.0:1789760784.673783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760784.673787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760784.673791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760784.673794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760784.673797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760784.673805:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760784.673812:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.673904:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:2.0:1789760784.674344:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72925184<>72929280 00000100:00100000:3.0:1789760784.674413:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760784.674716:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565509376:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.686287:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985112 gen 3 00000008:00010000:2.0:1789760784.686324:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760784.686331:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.686343:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.686429:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760784.686455:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565509376:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760784.687789:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17804, 17804] 00000080:00200000:3.0:1789760784.687807:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760784.687873:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.688135:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72929280 i_blocks 142464 00000020:00200000:2.0:1789760784.688141:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.688154:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.688185:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72929280, count: 4096 00000080:00200000:2.0:1789760784.688266:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72929280, bytes: 4096 00000080:00200000:2.0:1789760784.688278:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.688286:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.688288:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72929280, 72933376) 0 0 00000080:00200000:2.0:1789760784.688292:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.688306:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17805, 17805] 00020000:00200000:2.0:1789760784.688313:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72929280 chunk: [72929280, 72933376) 72933376, 4096 00020000:00200000:2.0:1789760784.688317:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.688334:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17805, 17805] 00000008:00010000:2.0:1789760784.688348:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.688353:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.688362:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72929280, 72933376) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.688373:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72929280=0x458d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.688420:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.688423:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72933376 00000020:00010000:2.0:1789760784.688429:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.688433:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.688435:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.688439:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.688441:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.688448:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.688454:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.688570:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.688691:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72929280<>72933376 00000100:00100000:3.0:1789760784.688760:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565509504:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.701265:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985113 gen 3 00000008:00010000:2.0:1789760784.701318:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:2.0:1789760784.701326:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.701338:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.701360:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:2.0:1789760784.701388:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565509504:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.701518:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17805, 17805] 00000080:00200000:0.0:1789760784.701537:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.701543:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.701556:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72933376 i_blocks 142472 00000020:00200000:0.0:1789760784.701560:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.701572:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.701600:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72933376, count: 4096 00000080:00200000:0.0:1789760784.701607:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72933376, bytes: 4096 00000080:00200000:0.0:1789760784.701619:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.701627:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.701629:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72933376, 72937472) 0 0 00000080:00200000:0.0:1789760784.701633:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.701679:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17806, 17806] 00020000:00200000:0.0:1789760784.701686:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72933376 chunk: [72933376, 72937472) 72937472, 4096 00020000:00200000:0.0:1789760784.701691:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.701707:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17806, 17806] 00000008:00010000:0.0:1789760784.701721:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.701726:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.701735:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72933376, 72937472) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.701746:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72933376=0x458e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.701794:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.701797:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72937472 00000020:00010000:0.0:1789760784.701804:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.701807:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.701810:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.701813:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.701816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.701823:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.701829:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.701847:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.701957:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72933376<>72937472 00000100:00100000:0.0:1789760784.702005:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760784.702036:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565509632:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.710275:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985114 gen 3 00000008:00010000:0.0:1789760784.710743:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.710753:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.710766:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.710856:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.710887:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565509632:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.713019:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17806, 17806] 00000080:00200000:2.0:1789760784.713045:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.713051:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.713070:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72937472 i_blocks 142480 00000020:00200000:2.0:1789760784.713077:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.713095:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.713129:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72937472, count: 4096 00000080:00200000:2.0:1789760784.713137:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72937472, bytes: 4096 00000080:00200000:2.0:1789760784.713151:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.713161:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.713164:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72937472, 72941568) 0 0 00000080:00200000:2.0:1789760784.713169:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.713186:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17807, 17807] 00020000:00200000:2.0:1789760784.713196:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72937472 chunk: [72937472, 72941568) 72941568, 4096 00020000:00200000:2.0:1789760784.713201:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.713225:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17807, 17807] 00000008:00010000:2.0:1789760784.714713:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.714727:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.714744:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72937472, 72941568) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.714762:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72937472=0x458f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.715211:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.715215:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72941568 00000020:00010000:2.0:1789760784.715533:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.715538:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.715542:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.715545:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.715548:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.715620:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.715631:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.716061:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760784.716206:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 72937472<>72941568 00000100:00100000:3.0:1789760784.716278:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760784.716697:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565509760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760784.730112:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985115 gen 3 00000008:00010000:3.0:1789760784.730161:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760784.730169:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760784.730181:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760784.730204:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760784.730245:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565509760:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.730404:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17807, 17807] 00000080:00200000:0.0:1789760784.730422:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.730427:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.730443:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72941568 i_blocks 142488 00000020:00200000:0.0:1789760784.730447:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.730457:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.730500:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72941568, count: 4096 00000080:00200000:0.0:1789760784.730516:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72941568, bytes: 4096 00000080:00200000:0.0:1789760784.730527:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.730536:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.730538:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72941568, 72945664) 0 0 00000080:00200000:0.0:1789760784.730542:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.730835:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17808, 17808] 00020000:00200000:0.0:1789760784.730844:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72941568 chunk: [72941568, 72945664) 72945664, 4096 00020000:00200000:0.0:1789760784.730849:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.730870:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17808, 17808] 00000008:00010000:0.0:1789760784.730891:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.730899:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.730911:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72941568, 72945664) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.730926:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72941568=0x4590000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.730988:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.730991:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72945664 00000020:00010000:0.0:1789760784.731000:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.731004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.731007:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.731011:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d06c8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.731013:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.731022:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.731029:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.731047:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.731175:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72941568<>72945664 00000100:00100000:1.0:1789760784.732418:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760784.732452:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565509888:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.750811:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985116 gen 3 00000008:00010000:0.0:1789760784.750859:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.750868:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.750879:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.750964:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.750996:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565509888:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760784.751542:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17808, 17808] 00000080:00200000:3.0:1789760784.751572:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760784.751579:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760784.751594:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72945664 i_blocks 142496 00000020:00200000:3.0:1789760784.751598:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760784.751611:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760784.751712:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72945664, count: 4096 00000080:00200000:3.0:1789760784.751722:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72945664, bytes: 4096 00000080:00200000:3.0:1789760784.751733:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760784.751742:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760784.751744:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72945664, 72949760) 0 0 00000080:00200000:3.0:1789760784.751749:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760784.751767:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17809, 17809] 00020000:00200000:3.0:1789760784.751775:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72945664 chunk: [72945664, 72949760) 72949760, 4096 00020000:00200000:3.0:1789760784.751779:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760784.751799:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17809, 17809] 00000008:00010000:3.0:1789760784.751817:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760784.751824:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760784.751837:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72945664, 72949760) append 0 append lockpos 0 00000080:00200000:3.0:1789760784.751852:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72945664=0x4591000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760784.751914:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760784.751917:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72949760 00000020:00010000:3.0:1789760784.751926:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760784.751930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760784.751933:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760784.751937:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760784.751940:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760784.751948:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760784.751955:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760784.751974:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760784.752112:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 72945664<>72949760 00000100:00100000:3.0:1789760784.752164:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565510016:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.769788:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985117 gen 3 00000008:00010000:2.0:1789760784.769848:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760784.769856:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.769868:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.769894:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760784.769923:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565510016:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.770208:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17809, 17809] 00000080:00200000:0.0:1789760784.770225:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.770232:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.770247:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72949760 i_blocks 142504 00000020:00200000:0.0:1789760784.770251:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.770262:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.770296:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72949760, count: 4096 00000080:00200000:0.0:1789760784.770303:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72949760, bytes: 4096 00000080:00200000:0.0:1789760784.770314:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.770324:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.770326:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72949760, 72953856) 0 0 00000080:00200000:0.0:1789760784.770331:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.770347:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17810, 17810] 00020000:00200000:0.0:1789760784.770356:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72949760 chunk: [72949760, 72953856) 72953856, 4096 00020000:00200000:0.0:1789760784.770361:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.770380:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17810, 17810] 00000008:00010000:0.0:1789760784.770399:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.770405:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.770417:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72949760, 72953856) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.770430:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72949760=0x4592000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.770621:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.770625:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72953856 00000020:00010000:0.0:1789760784.770634:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.770637:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.770640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.770688:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.770691:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.770699:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.770707:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.770724:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.770861:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72949760<>72953856 00000100:00100000:1.0:1789760784.772060:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565510144:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.789022:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985118 gen 3 00000008:00010000:0.0:1789760784.789072:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.789080:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.789088:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.789104:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.789131:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565510144:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760784.789137:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17810, 17810] 00000080:00200000:3.0:1789760784.789155:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760784.789161:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760784.789176:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72953856 i_blocks 142512 00000020:00200000:3.0:1789760784.789180:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760784.789192:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760784.789226:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72953856, count: 4096 00000080:00200000:3.0:1789760784.789234:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72953856, bytes: 4096 00000080:00200000:3.0:1789760784.789244:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760784.789253:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760784.789255:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72953856, 72957952) 0 0 00000080:00200000:3.0:1789760784.789259:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760784.789276:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17811, 17811] 00020000:00200000:3.0:1789760784.789286:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72953856 chunk: [72953856, 72957952) 72957952, 4096 00020000:00200000:3.0:1789760784.789291:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760784.789311:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17811, 17811] 00000008:00010000:3.0:1789760784.789326:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760784.789330:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760784.789341:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72953856, 72957952) append 0 append lockpos 0 00000080:00200000:3.0:1789760784.789352:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72953856=0x4593000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760784.789450:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760784.789453:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72957952 00000020:00010000:3.0:1789760784.789470:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760784.789474:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760784.789478:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760784.789482:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760784.789485:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760784.789494:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760784.789501:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760784.789517:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760784.789640:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 72953856<>72957952 00000100:00100000:2.0:1789760784.789760:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565510272:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.799235:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985119 gen 3 00000008:00010000:2.0:1789760784.799283:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760784.799292:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.799304:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.799427:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760784.799459:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565510272:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760784.799747:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17811, 17811] 00000080:00200000:3.0:1789760784.799764:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760784.799770:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.799934:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72957952 i_blocks 142520 00000020:00200000:2.0:1789760784.799939:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.799950:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.799984:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72957952, count: 4096 00000080:00200000:2.0:1789760784.799992:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72957952, bytes: 4096 00000080:00200000:2.0:1789760784.800004:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.800106:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.800109:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72957952, 72962048) 0 0 00000080:00200000:2.0:1789760784.800115:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.800131:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17812, 17812] 00020000:00200000:2.0:1789760784.800139:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72957952 chunk: [72957952, 72962048) 72962048, 4096 00020000:00200000:2.0:1789760784.800143:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.800164:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17812, 17812] 00000008:00010000:2.0:1789760784.800181:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.800187:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.800199:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72957952, 72962048) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.800213:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72957952=0x4594000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.800268:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.800271:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72962048 00000020:00010000:2.0:1789760784.800280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.800283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.800286:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.800289:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.800292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.800300:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.800307:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.800480:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760784.800756:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 72957952<>72962048 00000100:00100000:3.0:1789760784.800906:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565510400:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.814984:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985120 gen 3 00000008:00010000:2.0:1789760784.815036:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760784.815044:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.815057:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.815083:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760784.815113:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565510400:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.815236:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17812, 17812] 00000080:00200000:0.0:1789760784.815253:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.815259:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.815275:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72962048 i_blocks 142528 00000020:00200000:0.0:1789760784.815278:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.815288:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.815325:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72962048, count: 4096 00000080:00200000:0.0:1789760784.815333:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72962048, bytes: 4096 00000080:00200000:0.0:1789760784.815349:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.815358:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.815361:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72962048, 72966144) 0 0 00000080:00200000:0.0:1789760784.815365:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.815381:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17813, 17813] 00020000:00200000:0.0:1789760784.815390:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72962048 chunk: [72962048, 72966144) 72966144, 4096 00020000:00200000:0.0:1789760784.815394:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.815420:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17813, 17813] 00000008:00010000:0.0:1789760784.815438:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.815443:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.815455:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72962048, 72966144) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.815478:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72962048=0x4595000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.815546:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.815549:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72966144 00000020:00010000:0.0:1789760784.815557:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.815561:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.815564:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.815568:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.815571:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.815579:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.815586:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.815603:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.815763:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72962048<>72966144 00000100:00100000:0.0:1789760784.815799:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760784.815816:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565510528:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.828394:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985121 gen 3 00000008:00010000:0.0:1789760784.828515:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.828524:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.828535:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.828563:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.828594:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565510528:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.828780:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17813, 17813] 00000080:00200000:2.0:1789760784.828797:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.828803:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.828919:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72966144 i_blocks 142536 00000020:00200000:2.0:1789760784.828923:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.828935:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.829039:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72966144, count: 4096 00000080:00200000:2.0:1789760784.829048:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72966144, bytes: 4096 00000080:00200000:2.0:1789760784.829059:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.829068:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.829071:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72966144, 72970240) 0 0 00000080:00200000:2.0:1789760784.829076:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.829092:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17814, 17814] 00020000:00200000:2.0:1789760784.829100:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72966144 chunk: [72966144, 72970240) 72970240, 4096 00020000:00200000:2.0:1789760784.829105:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.829123:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17814, 17814] 00000008:00010000:2.0:1789760784.829141:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.829147:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.829157:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72966144, 72970240) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.829171:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72966144=0x4596000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.829232:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.829235:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72970240 00000020:00010000:2.0:1789760784.829244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.829247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.829250:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.829254:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.829257:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.829264:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.829271:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.829290:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760784.831986:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 72966144<>72970240 00000100:00100000:3.0:1789760784.832078:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760784.832248:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565510656:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760784.845803:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985122 gen 3 00000008:00010000:2.0:1789760784.845868:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760784.845876:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760784.845886:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760784.845920:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760784.845950:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565510656:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760784.846614:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17814, 17814] 00000080:00200000:0.0:1789760784.846691:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760784.846699:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760784.846715:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72970240 i_blocks 142544 00000020:00200000:0.0:1789760784.846719:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760784.846730:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760784.846767:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72970240, count: 4096 00000080:00200000:0.0:1789760784.846775:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72970240, bytes: 4096 00000080:00200000:0.0:1789760784.846788:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760784.846796:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760784.846799:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72970240, 72974336) 0 0 00000080:00200000:0.0:1789760784.846813:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760784.846830:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17815, 17815] 00020000:00200000:0.0:1789760784.846840:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72970240 chunk: [72970240, 72974336) 72974336, 4096 00020000:00200000:0.0:1789760784.846844:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760784.846867:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17815, 17815] 00000008:00010000:0.0:1789760784.846889:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760784.846895:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760784.846907:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72970240, 72974336) append 0 append lockpos 0 00000080:00200000:0.0:1789760784.846921:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72970240=0x4597000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760784.846985:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760784.846988:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72974336 00000020:00010000:0.0:1789760784.846996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760784.846999:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.847002:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760784.847006:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760784.847008:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760784.847118:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760784.847126:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760784.847147:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760784.847273:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 72970240<>72974336 00000100:00100000:1.0:1789760784.847396:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760784.847429:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565510784:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.857913:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985123 gen 3 00000008:00010000:0.0:1789760784.857961:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760784.857970:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.857982:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.858007:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760784.858038:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565510784:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760784.858148:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17815, 17815] 00000080:00200000:1.0:1789760784.858165:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760784.858171:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760784.858187:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72974336 i_blocks 142552 00000020:00200000:1.0:1789760784.858191:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760784.858202:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760784.858238:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72974336, count: 4096 00000080:00200000:1.0:1789760784.858245:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72974336, bytes: 4096 00000080:00200000:1.0:1789760784.858255:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760784.858263:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760784.858266:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72974336, 72978432) 0 0 00000080:00200000:1.0:1789760784.858271:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760784.858286:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17816, 17816] 00020000:00200000:1.0:1789760784.858295:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72974336 chunk: [72974336, 72978432) 72978432, 4096 00020000:00200000:1.0:1789760784.858300:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760784.858318:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17816, 17816] 00000008:00010000:1.0:1789760784.858336:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760784.858341:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760784.858353:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72974336, 72978432) append 0 append lockpos 0 00000080:00200000:1.0:1789760784.858366:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72974336=0x4598000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760784.859199:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760784.859205:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72978432 00000020:00010000:1.0:1789760784.859217:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760784.859221:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.859225:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.859228:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760784.859231:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760784.859240:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760784.859248:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760784.859269:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760784.859689:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 72974336<>72978432 00000100:00100000:0.0:1789760784.859762:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565510912:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.869993:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985124 gen 3 00000008:00010000:0.0:1789760784.870045:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760784.870054:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.870066:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.870966:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760784.871022:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565510912:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760784.872142:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17816, 17816] 00000080:00200000:1.0:1789760784.872164:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760784.872170:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760784.872188:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72978432 i_blocks 142560 00000020:00200000:1.0:1789760784.872193:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760784.872206:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760784.872242:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72978432, count: 4096 00000080:00200000:1.0:1789760784.872250:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72978432, bytes: 4096 00000080:00200000:1.0:1789760784.872266:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760784.872276:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760784.872279:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72978432, 72982528) 0 0 00000080:00200000:1.0:1789760784.872284:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760784.872302:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17817, 17817] 00020000:00200000:1.0:1789760784.872311:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72978432 chunk: [72978432, 72982528) 72982528, 4096 00020000:00200000:1.0:1789760784.872316:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760784.872339:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17817, 17817] 00000008:00010000:1.0:1789760784.872371:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760784.872377:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760784.872391:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72978432, 72982528) append 0 append lockpos 0 00000080:00200000:1.0:1789760784.872529:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72978432=0x4599000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760784.872616:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760784.872620:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72982528 00000020:00010000:1.0:1789760784.872629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760784.872632:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.872636:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.872640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760784.872671:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760784.872680:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760784.872687:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760784.872713:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760784.873730:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 72978432<>72982528 00000100:00100000:1.0:1789760784.874141:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565511040:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.882741:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985125 gen 3 00000008:00010000:0.0:1789760784.882789:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760784.882797:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.882808:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.882861:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760784.882890:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565511040:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760784.882959:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17817, 17817] 00000080:00200000:1.0:1789760784.882975:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760784.882980:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760784.882995:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72982528 i_blocks 142568 00000020:00200000:1.0:1789760784.882999:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760784.883007:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760784.883036:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72982528, count: 4096 00000080:00200000:1.0:1789760784.883043:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72982528, bytes: 4096 00000080:00200000:1.0:1789760784.883052:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760784.883060:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760784.883064:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72982528, 72986624) 0 0 00000080:00200000:1.0:1789760784.883069:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760784.883084:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17818, 17818] 00020000:00200000:1.0:1789760784.883092:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72982528 chunk: [72982528, 72986624) 72986624, 4096 00020000:00200000:1.0:1789760784.883097:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760784.883113:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17818, 17818] 00000008:00010000:1.0:1789760784.883130:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760784.883135:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760784.883145:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72982528, 72986624) append 0 append lockpos 0 00000080:00200000:1.0:1789760784.883158:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72982528=0x459a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760784.883209:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760784.883212:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72986624 00000020:00010000:1.0:1789760784.883219:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760784.883223:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.883346:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.883350:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760784.883353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760784.883360:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760784.883367:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760784.883383:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760784.883506:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 72982528<>72986624 00000100:00100000:0.0:1789760784.883631:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565511168:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.892699:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985126 gen 3 00000008:00010000:0.0:1789760784.892753:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760784.892762:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.892773:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.892786:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760784.892825:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565511168:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760784.892979:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17818, 17818] 00000080:00200000:1.0:1789760784.892996:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760784.893014:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760784.893028:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72986624 i_blocks 142576 00000020:00200000:1.0:1789760784.893032:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760784.893042:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760784.893076:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72986624, count: 4096 00000080:00200000:1.0:1789760784.893084:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72986624, bytes: 4096 00000080:00200000:1.0:1789760784.893095:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760784.893104:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760784.893108:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72986624, 72990720) 0 0 00000080:00200000:1.0:1789760784.893112:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760784.893129:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17819, 17819] 00020000:00200000:1.0:1789760784.893138:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72986624 chunk: [72986624, 72990720) 72990720, 4096 00020000:00200000:1.0:1789760784.893142:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760784.893163:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17819, 17819] 00000008:00010000:1.0:1789760784.893180:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760784.893187:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760784.893198:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72986624, 72990720) append 0 append lockpos 0 00000080:00200000:1.0:1789760784.893211:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72986624=0x459b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760784.893269:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760784.893272:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72990720 00000020:00010000:1.0:1789760784.893280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760784.893284:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.893287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.893291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760784.893293:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760784.893300:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760784.893307:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760784.893323:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760784.893761:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 72986624<>72990720 00000100:00100000:0.0:1789760784.895144:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760784.895178:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565511296:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.921007:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985127 gen 3 00000008:00010000:0.0:1789760784.921057:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760784.921066:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.921078:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.921102:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760784.921130:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565511296:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760784.921230:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17819, 17819] 00000080:00200000:1.0:1789760784.921248:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760784.921255:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760784.921271:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72990720 i_blocks 142584 00000020:00200000:1.0:1789760784.921276:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760784.921286:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760784.921319:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72990720, count: 4096 00000080:00200000:1.0:1789760784.921328:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72990720, bytes: 4096 00000080:00200000:1.0:1789760784.921337:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760784.921350:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760784.921354:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72990720, 72994816) 0 0 00000080:00200000:1.0:1789760784.921359:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760784.921376:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17820, 17820] 00020000:00200000:1.0:1789760784.921385:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72990720 chunk: [72990720, 72994816) 72994816, 4096 00020000:00200000:1.0:1789760784.921391:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760784.921421:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17820, 17820] 00000008:00010000:1.0:1789760784.921438:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760784.921443:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760784.921455:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72990720, 72994816) append 0 append lockpos 0 00000080:00200000:1.0:1789760784.921480:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72990720=0x459c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760784.921540:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760784.921543:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72994816 00000020:00010000:1.0:1789760784.921551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760784.921555:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.921558:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.921562:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760784.921565:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760784.921572:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760784.921579:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760784.921596:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760784.921774:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 72990720<>72994816 00000100:00100000:0.0:1789760784.923179:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565511424:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760784.925707:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760784.948727:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985128 gen 3 00000008:00010000:0.0:1789760784.948780:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760784.948789:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.948802:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.948817:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760784.948841:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565511424:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760784.949600:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17820, 17820] 00000080:00200000:1.0:1789760784.949620:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760784.949627:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760784.949671:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72994816 i_blocks 142592 00000020:00200000:1.0:1789760784.949677:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760784.949690:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760784.949730:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72994816, count: 4096 00000080:00200000:1.0:1789760784.949738:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72994816, bytes: 4096 00000080:00200000:1.0:1789760784.949753:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760784.949762:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760784.949766:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72994816, 72998912) 0 0 00000080:00200000:1.0:1789760784.949771:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760784.949789:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17821, 17821] 00020000:00200000:1.0:1789760784.949798:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72994816 chunk: [72994816, 72998912) 72998912, 4096 00020000:00200000:1.0:1789760784.949803:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760784.949824:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17821, 17821] 00000008:00010000:1.0:1789760784.949844:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760784.949850:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760784.949861:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72994816, 72998912) append 0 append lockpos 0 00000080:00200000:1.0:1789760784.949876:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72994816=0x459d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760784.949937:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760784.949940:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 72998912 00000020:00010000:1.0:1789760784.949948:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760784.949952:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.949954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.949957:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760784.949960:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760784.949968:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760784.949975:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760784.949995:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760784.950119:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 72994816<>72998912 00000100:00100000:0.0:1789760784.950243:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760784.950266:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565511552:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.967035:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985129 gen 3 00000008:00010000:0.0:1789760784.967087:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760784.967096:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.967106:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.967127:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760784.967156:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565511552:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760784.968689:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17821, 17821] 00000080:00200000:1.0:1789760784.968716:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760784.968724:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760784.968742:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 72998912 i_blocks 142600 00000020:00200000:1.0:1789760784.968748:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760784.968763:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760784.968864:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 72998912, count: 4096 00000080:00200000:1.0:1789760784.968874:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 72998912, bytes: 4096 00000080:00200000:1.0:1789760784.968889:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760784.968899:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760784.968903:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [72998912, 73003008) 0 0 00000080:00200000:1.0:1789760784.968908:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760784.968929:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17822, 17822] 00020000:00200000:1.0:1789760784.968941:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 72998912 chunk: [72998912, 73003008) 73003008, 4096 00020000:00200000:1.0:1789760784.968947:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760784.968975:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17822, 17822] 00000008:00010000:1.0:1789760784.968997:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760784.969005:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760784.969018:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [72998912, 73003008) append 0 append lockpos 0 00000080:00200000:1.0:1789760784.969032:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=72998912=0x459e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760784.969101:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760784.969104:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73003008 00000020:00010000:1.0:1789760784.969112:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760784.969116:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.969118:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.969122:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760784.969125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760784.969133:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760784.969140:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760784.969161:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760784.969306:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 72998912<>73003008 00000100:00100000:0.0:1789760784.969597:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565511680:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.981706:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985130 gen 3 00000008:00010000:0.0:1789760784.981762:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760784.981772:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.981783:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.981796:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760784.981835:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565511680:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760784.982000:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17822, 17822] 00000080:00200000:1.0:1789760784.982018:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760784.982024:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760784.982040:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73003008 i_blocks 142608 00000020:00200000:1.0:1789760784.982044:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760784.982054:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760784.982090:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73003008, count: 4096 00000080:00200000:1.0:1789760784.982099:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73003008, bytes: 4096 00000080:00200000:1.0:1789760784.982111:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760784.982121:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760784.982125:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73003008, 73007104) 0 0 00000080:00200000:1.0:1789760784.982144:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760784.982162:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17823, 17823] 00020000:00200000:1.0:1789760784.982173:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73003008 chunk: [73003008, 73007104) 73007104, 4096 00020000:00200000:1.0:1789760784.982178:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760784.982203:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17823, 17823] 00000008:00010000:1.0:1789760784.982285:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760784.982292:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760784.982304:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73003008, 73007104) append 0 append lockpos 0 00000080:00200000:1.0:1789760784.982320:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73003008=0x459f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760784.982593:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760784.982596:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73007104 00000020:00010000:1.0:1789760784.982606:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:1.0:1789760784.982610:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.982612:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760784.982616:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760784.982619:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760784.982626:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760784.982632:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760784.982683:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760784.983275:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 73003008<>73007104 00000100:00100000:0.0:1789760784.983547:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565511808:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760784.993738:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985131 gen 3 00000008:00010000:0.0:1789760784.993788:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760784.993797:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760784.993808:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760784.993841:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760784.993873:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565511808:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760784.993885:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17823, 17823] 00000080:00200000:2.0:1789760784.993904:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760784.993909:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760784.993925:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73007104 i_blocks 142616 00000020:00200000:2.0:1789760784.993929:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760784.993939:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760784.993975:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73007104, count: 4096 00000080:00200000:2.0:1789760784.993983:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73007104, bytes: 4096 00000080:00200000:2.0:1789760784.993993:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760784.994001:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760784.994004:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73007104, 73011200) 0 0 00000080:00200000:2.0:1789760784.994009:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760784.994025:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17824, 17824] 00020000:00200000:2.0:1789760784.994035:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73007104 chunk: [73007104, 73011200) 73011200, 4096 00020000:00200000:2.0:1789760784.994041:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760784.994059:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17824, 17824] 00000008:00010000:2.0:1789760784.994077:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760784.994084:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760784.994096:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73007104, 73011200) append 0 append lockpos 0 00000080:00200000:2.0:1789760784.994109:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73007104=0x45a0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760784.994169:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760784.994172:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73011200 00000020:00010000:2.0:1789760784.994181:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760784.994184:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.994188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760784.994192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760784.994194:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760784.994201:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760784.994208:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760784.994223:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760784.994347:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 73007104<>73011200 00000100:00100000:2.0:1789760784.994401:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760784.994417:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565511936:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.010726:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985132 gen 3 00000008:00010000:2.0:1789760785.010785:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760785.010795:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.010815:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.010881:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760785.010912:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565511936:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.011878:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17824, 17824] 00000080:00200000:3.0:1789760785.011896:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.011903:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.012104:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73011200 i_blocks 142624 00000020:00200000:2.0:1789760785.012110:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.012121:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.012292:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73011200, count: 4096 00000080:00200000:2.0:1789760785.012301:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73011200, bytes: 4096 00000080:00200000:2.0:1789760785.012314:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.012323:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.012326:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73011200, 73015296) 0 0 00000080:00200000:2.0:1789760785.012331:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.012482:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17825, 17825] 00020000:00200000:2.0:1789760785.012553:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73011200 chunk: [73011200, 73015296) 73015296, 4096 00020000:00200000:2.0:1789760785.012558:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.012581:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17825, 17825] 00000008:00010000:2.0:1789760785.012603:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.012609:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.012621:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73011200, 73015296) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.012639:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73011200=0x45a1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.012730:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.012733:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73015296 00000020:00010000:2.0:1789760785.012741:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.012745:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.012748:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.012751:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.012754:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.012762:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.012769:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.012898:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760785.013024:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 73011200<>73015296 00000100:00100000:3.0:1789760785.013106:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512064:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.025387:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985133 gen 3 00000008:00010000:2.0:1789760785.025952:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760785.025960:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.025972:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.025997:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760785.026028:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512064:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760785.026054:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17825, 17825] 00000080:00200000:0.0:1789760785.026075:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760785.026091:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760785.026107:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73015296 i_blocks 142632 00000020:00200000:0.0:1789760785.026112:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760785.026123:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760785.026167:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73015296, count: 4096 00000080:00200000:0.0:1789760785.026175:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73015296, bytes: 4096 00000080:00200000:0.0:1789760785.026194:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760785.026204:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760785.026207:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73015296, 73019392) 0 0 00000080:00200000:0.0:1789760785.026211:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760785.026227:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17826, 17826] 00020000:00200000:0.0:1789760785.026236:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73015296 chunk: [73015296, 73019392) 73019392, 4096 00020000:00200000:0.0:1789760785.026242:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760785.026261:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17826, 17826] 00000008:00010000:0.0:1789760785.026280:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760785.026286:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760785.026298:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73015296, 73019392) append 0 append lockpos 0 00000080:00200000:0.0:1789760785.026311:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73015296=0x45a2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760785.026390:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760785.026393:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73019392 00000020:00010000:0.0:1789760785.026401:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760785.026404:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.026407:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.026411:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760785.026413:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760785.026421:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760785.026427:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760785.026444:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760785.026573:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 73015296<>73019392 00000100:00100000:0.0:1789760785.026619:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760785.026637:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565512192:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760785.039014:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985134 gen 3 00000008:00010000:0.0:1789760785.039064:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760785.039073:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760785.039084:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760785.039119:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760785.039159:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565512192:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.039456:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17826, 17826] 00000080:00200000:2.0:1789760785.039483:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.039489:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.039504:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73019392 i_blocks 142640 00000020:00200000:2.0:1789760785.039507:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.039531:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.039565:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73019392, count: 4096 00000080:00200000:2.0:1789760785.039572:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73019392, bytes: 4096 00000080:00200000:2.0:1789760785.039585:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.039594:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.039597:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73019392, 73023488) 0 0 00000080:00200000:2.0:1789760785.039601:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.039617:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17827, 17827] 00020000:00200000:2.0:1789760785.039625:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73019392 chunk: [73019392, 73023488) 73023488, 4096 00020000:00200000:2.0:1789760785.039630:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.039688:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17827, 17827] 00000008:00010000:2.0:1789760785.039705:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.039711:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.039722:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73019392, 73023488) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.039735:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73019392=0x45a3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.039789:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.039791:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73023488 00000020:00010000:2.0:1789760785.039800:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.039804:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.039807:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.039811:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.039814:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.039822:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.039829:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.039846:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760785.039966:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 73019392<>73023488 00000100:00100000:2.0:1789760785.040174:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760785.040254:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512320:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.059920:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985135 gen 3 00000008:00010000:2.0:1789760785.059973:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760785.059982:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.059994:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.060020:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760785.060053:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512320:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.060171:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17827, 17827] 00000080:00200000:3.0:1789760785.060231:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.060237:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.060253:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73023488 i_blocks 142648 00000020:00200000:3.0:1789760785.060256:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.060265:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.060308:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73023488, count: 4096 00000080:00200000:3.0:1789760785.060316:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73023488, bytes: 4096 00000080:00200000:3.0:1789760785.060326:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.060335:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.060337:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73023488, 73027584) 0 0 00000080:00200000:3.0:1789760785.060341:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.060359:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17828, 17828] 00020000:00200000:3.0:1789760785.060367:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73023488 chunk: [73023488, 73027584) 73027584, 4096 00020000:00200000:3.0:1789760785.060372:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.060392:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17828, 17828] 00000008:00010000:3.0:1789760785.060409:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.060415:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.060439:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73023488, 73027584) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.060454:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73023488=0x45a4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.060523:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.060526:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73027584 00000020:00010000:3.0:1789760785.060534:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.060539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.060542:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.060545:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.060548:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.060556:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.060563:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.060580:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760785.060916:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 73023488<>73027584 00000100:00100000:2.0:1789760785.061871:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512448:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.072616:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985136 gen 3 00000008:00010000:2.0:1789760785.072717:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760785.072725:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.072736:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.072757:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:2.0:1789760785.072786:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512448:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.073034:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17828, 17828] 00000080:00200000:3.0:1789760785.073053:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.073060:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.073076:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73027584 i_blocks 142656 00000020:00200000:3.0:1789760785.073079:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.073089:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.073125:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73027584, count: 4096 00000080:00200000:3.0:1789760785.073133:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73027584, bytes: 4096 00000080:00200000:3.0:1789760785.073145:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.073155:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.073157:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73027584, 73031680) 0 0 00000080:00200000:3.0:1789760785.073162:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.073181:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17829, 17829] 00020000:00200000:3.0:1789760785.073202:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73027584 chunk: [73027584, 73031680) 73031680, 4096 00020000:00200000:3.0:1789760785.073208:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.073230:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17829, 17829] 00000008:00010000:3.0:1789760785.073248:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.073255:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.073267:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73027584, 73031680) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.073282:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73027584=0x45a5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.073573:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.073577:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73031680 00000020:00010000:3.0:1789760785.073586:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.073591:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.073593:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.073597:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.073600:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.073610:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.073617:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.073636:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760785.073813:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 73027584<>73031680 00000100:00100000:2.0:1789760785.074080:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760785.074101:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512576:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.088145:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985137 gen 3 00000008:00010000:2.0:1789760785.088196:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760785.088205:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.088226:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.088247:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760785.088277:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512576:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.088451:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17829, 17829] 00000080:00200000:3.0:1789760785.088477:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.088483:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.088499:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73031680 i_blocks 142664 00000020:00200000:3.0:1789760785.088502:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.088513:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.088549:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73031680, count: 4096 00000080:00200000:3.0:1789760785.088556:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73031680, bytes: 4096 00000080:00200000:3.0:1789760785.088567:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.088576:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.088579:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73031680, 73035776) 0 0 00000080:00200000:3.0:1789760785.088584:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.088602:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17830, 17830] 00020000:00200000:3.0:1789760785.088613:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73031680 chunk: [73031680, 73035776) 73035776, 4096 00020000:00200000:3.0:1789760785.088618:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.088799:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17830, 17830] 00000008:00010000:3.0:1789760785.088817:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.088823:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.088835:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73031680, 73035776) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.088848:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73031680=0x45a6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.088906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.088909:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73035776 00000020:00010000:3.0:1789760785.088917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.088996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.089000:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.089004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.089007:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.089016:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.089024:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.089043:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760785.089165:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 73031680<>73035776 00000100:00100000:2.0:1789760785.089248:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512704:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.101291:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985138 gen 3 00000008:00010000:2.0:1789760785.101339:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760785.101347:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.101359:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.101442:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760785.101489:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512704:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.101873:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17830, 17830] 00000080:00200000:3.0:1789760785.101891:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.101897:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.101914:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73035776 i_blocks 142672 00000020:00200000:3.0:1789760785.101918:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.101928:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.101963:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73035776, count: 4096 00000080:00200000:3.0:1789760785.101970:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73035776, bytes: 4096 00000080:00200000:3.0:1789760785.101981:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.101989:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.101992:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73035776, 73039872) 0 0 00000080:00200000:3.0:1789760785.101997:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.102028:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17831, 17831] 00020000:00200000:3.0:1789760785.102038:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73035776 chunk: [73035776, 73039872) 73039872, 4096 00020000:00200000:3.0:1789760785.102042:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.102062:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17831, 17831] 00000008:00010000:3.0:1789760785.102078:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.102084:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.102095:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73035776, 73039872) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.102109:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73035776=0x45a7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.102356:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.102359:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73039872 00000020:00010000:3.0:1789760785.102368:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.102372:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.102376:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.102379:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.102382:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.102391:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.102398:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.102421:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760785.102556:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 73035776<>73039872 00000100:00100000:2.0:1789760785.102668:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565512832:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.116383:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985139 gen 3 00000008:00010000:2.0:1789760785.116436:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760785.116454:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.116474:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.116503:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:2.0:1789760785.116533:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565512832:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.116771:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17831, 17831] 00000080:00200000:3.0:1789760785.116788:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.116795:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.116822:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73039872 i_blocks 142680 00000020:00200000:3.0:1789760785.116827:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.116838:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.116872:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73039872, count: 4096 00000080:00200000:3.0:1789760785.116880:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73039872, bytes: 4096 00000080:00200000:3.0:1789760785.116891:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.116898:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.116901:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73039872, 73043968) 0 0 00000080:00200000:3.0:1789760785.116905:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.116922:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17832, 17832] 00020000:00200000:3.0:1789760785.116931:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73039872 chunk: [73039872, 73043968) 73043968, 4096 00020000:00200000:3.0:1789760785.116936:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.116956:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17832, 17832] 00000008:00010000:3.0:1789760785.116975:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.116981:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.116993:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73039872, 73043968) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.117008:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73039872=0x45a8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.117066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.117069:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73043968 00000020:00010000:3.0:1789760785.117077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.117080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.117084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.117088:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.117091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.117099:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.117106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.117121:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760785.117323:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 73039872<>73043968 00000100:00100000:2.0:1789760785.117702:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512960:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.125098:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985140 gen 3 00000008:00010000:2.0:1789760785.125219:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760785.125227:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.125238:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.125259:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760785.125288:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565512960:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.125405:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17832, 17832] 00000080:00200000:3.0:1789760785.125424:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.125430:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.125445:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73043968 i_blocks 142688 00000020:00200000:3.0:1789760785.125449:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.125469:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.125511:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73043968, count: 4096 00000080:00200000:3.0:1789760785.125519:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73043968, bytes: 4096 00000080:00200000:3.0:1789760785.125530:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.125538:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.125541:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73043968, 73048064) 0 0 00000080:00200000:3.0:1789760785.125545:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.125562:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17833, 17833] 00020000:00200000:3.0:1789760785.125572:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73043968 chunk: [73043968, 73048064) 73048064, 4096 00020000:00200000:3.0:1789760785.125576:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.125595:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17833, 17833] 00000008:00010000:3.0:1789760785.125612:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.125618:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.125630:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73043968, 73048064) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.125687:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73043968=0x45a9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.125746:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.125749:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73048064 00000020:00010000:3.0:1789760785.125758:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.125763:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.125766:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.125769:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.125772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.125780:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.125786:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.125801:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760785.125922:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 73043968<>73048064 00000100:00100000:2.0:1789760785.126262:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760785.126281:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565513088:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.137493:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985141 gen 3 00000008:00010000:2.0:1789760785.137540:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760785.137548:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.137559:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.137580:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760785.137609:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565513088:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.138004:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17833, 17833] 00000080:00200000:3.0:1789760785.138023:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.138028:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.138043:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73048064 i_blocks 142696 00000020:00200000:3.0:1789760785.138047:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.138058:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.138093:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73048064, count: 4096 00000080:00200000:3.0:1789760785.138102:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73048064, bytes: 4096 00000080:00200000:3.0:1789760785.138113:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.138122:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.138125:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73048064, 73052160) 0 0 00000080:00200000:3.0:1789760785.138130:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.138269:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17834, 17834] 00020000:00200000:3.0:1789760785.138279:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73048064 chunk: [73048064, 73052160) 73052160, 4096 00020000:00200000:3.0:1789760785.138283:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.138305:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17834, 17834] 00000008:00010000:3.0:1789760785.138325:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.138331:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.138342:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73048064, 73052160) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.138355:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73048064=0x45aa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.138911:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.138915:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73052160 00000020:00010000:3.0:1789760785.138926:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.138931:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.138934:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.138938:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.138941:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.138952:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.138959:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.138979:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760785.139118:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 73048064<>73052160 00000100:00100000:2.0:1789760785.139267:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565513216:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.158576:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985142 gen 3 00000008:00010000:2.0:1789760785.158641:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760785.158695:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.158706:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.158725:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760785.158755:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565513216:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.158775:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17834, 17834] 00000080:00200000:3.0:1789760785.158794:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.158800:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.158816:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73052160 i_blocks 142704 00000020:00200000:3.0:1789760785.158820:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.158831:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.158865:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73052160, count: 4096 00000080:00200000:3.0:1789760785.158873:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73052160, bytes: 4096 00000080:00200000:3.0:1789760785.158883:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.158892:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.158895:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73052160, 73056256) 0 0 00000080:00200000:3.0:1789760785.158899:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.158918:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17835, 17835] 00020000:00200000:3.0:1789760785.158927:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73052160 chunk: [73052160, 73056256) 73056256, 4096 00020000:00200000:3.0:1789760785.158932:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.158952:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17835, 17835] 00000008:00010000:3.0:1789760785.158970:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.158976:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.158989:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73052160, 73056256) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.159003:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73052160=0x45ab000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.159066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.159070:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73056256 00000020:00010000:3.0:1789760785.159079:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.159084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.159087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.159091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.159094:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.159102:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.159110:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.159128:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760785.160418:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 73052160<>73056256 00000100:00100000:3.0:1789760785.160508:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565513344:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.174207:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985143 gen 3 00000008:00010000:2.0:1789760785.174251:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760785.174258:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.174269:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.174320:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760785.174344:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565513344:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760785.174416:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17835, 17835] 00000080:00200000:0.0:1789760785.174547:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760785.174554:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760785.174577:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73056256 i_blocks 142712 00000020:00200000:0.0:1789760785.174581:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760785.174592:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760785.174631:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73056256, count: 4096 00000080:00200000:0.0:1789760785.174639:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73056256, bytes: 4096 00000080:00200000:0.0:1789760785.174912:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760785.174923:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760785.174925:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73056256, 73060352) 0 0 00000080:00200000:0.0:1789760785.174931:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760785.174948:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17836, 17836] 00020000:00200000:0.0:1789760785.174958:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73056256 chunk: [73056256, 73060352) 73060352, 4096 00020000:00200000:0.0:1789760785.174963:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760785.174986:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17836, 17836] 00000008:00010000:0.0:1789760785.175007:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760785.175014:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760785.175028:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73056256, 73060352) append 0 append lockpos 0 00000080:00200000:0.0:1789760785.175043:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73056256=0x45ac000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760785.175105:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760785.175108:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73060352 00000020:00010000:0.0:1789760785.175117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760785.175121:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.175124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.175128:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760785.175132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760785.175142:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760785.175149:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760785.175170:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760785.175299:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 73056256<>73060352 00000100:00100000:1.0:1789760785.175546:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760785.175566:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565513472:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760785.187014:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985144 gen 3 00000008:00010000:0.0:1789760785.187067:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760785.187076:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760785.187086:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760785.187156:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760785.187187:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565513472:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760785.187334:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17836, 17836] 00000080:00200000:1.0:1789760785.187354:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.187360:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760785.187375:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73060352 i_blocks 142720 00000020:00200000:1.0:1789760785.187378:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.187391:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.187426:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73060352, count: 4096 00000080:00200000:1.0:1789760785.187435:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73060352, bytes: 4096 00000080:00200000:1.0:1789760785.187446:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.187455:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.187469:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73060352, 73064448) 0 0 00000080:00200000:1.0:1789760785.187474:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.187491:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17837, 17837] 00020000:00200000:1.0:1789760785.187577:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73060352 chunk: [73060352, 73064448) 73064448, 4096 00020000:00200000:1.0:1789760785.187582:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.187604:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17837, 17837] 00000008:00010000:1.0:1789760785.187623:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.187629:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.187641:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73060352, 73064448) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.187818:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73060352=0x45ad000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.187882:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.187885:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73064448 00000020:00010000:1.0:1789760785.187894:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.187898:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.187902:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.187906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.187909:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.187917:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.187925:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.187946:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760785.188070:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 73060352<>73064448 00000100:00100000:1.0:1789760785.188119:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760785.188140:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565513600:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760785.202494:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985145 gen 3 00000008:00010000:0.0:1789760785.202543:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760785.202551:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760785.202562:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760785.202582:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760785.202610:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565513600:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760785.203058:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17837, 17837] 00000080:00200000:1.0:1789760785.203079:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.203086:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760785.203119:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73064448 i_blocks 142728 00000020:00200000:1.0:1789760785.203126:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.203139:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.203178:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73064448, count: 4096 00000080:00200000:1.0:1789760785.203198:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73064448, bytes: 4096 00000080:00200000:1.0:1789760785.203210:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.203220:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.203222:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73064448, 73068544) 0 0 00000080:00200000:1.0:1789760785.203227:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.203247:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17838, 17838] 00020000:00200000:1.0:1789760785.203256:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73064448 chunk: [73064448, 73068544) 73068544, 4096 00020000:00200000:1.0:1789760785.203261:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.203293:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17838, 17838] 00000008:00010000:1.0:1789760785.203310:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.203317:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.203330:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73064448, 73068544) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.203344:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73064448=0x45ae000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.203415:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.203418:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73068544 00000020:00010000:1.0:1789760785.203427:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.203430:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.203434:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.203437:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.203440:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.203448:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.203455:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.203814:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760785.203957:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 73064448<>73068544 00000100:00100000:1.0:1789760785.204008:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760785.204033:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565513728:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760785.221220:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985146 gen 3 00000008:00010000:0.0:1789760785.221274:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760785.221293:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760785.221304:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760785.221331:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760785.221364:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565513728:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.221505:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17838, 17838] 00000080:00200000:2.0:1789760785.221523:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.221529:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.221544:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73068544 i_blocks 142736 00000020:00200000:2.0:1789760785.221548:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.221568:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.221604:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73068544, count: 4096 00000080:00200000:2.0:1789760785.221612:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73068544, bytes: 4096 00000080:00200000:2.0:1789760785.221622:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.221631:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.221633:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73068544, 73072640) 0 0 00000080:00200000:2.0:1789760785.221637:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.221698:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17839, 17839] 00020000:00200000:2.0:1789760785.221708:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73068544 chunk: [73068544, 73072640) 73072640, 4096 00020000:00200000:2.0:1789760785.221713:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.221733:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17839, 17839] 00000008:00010000:2.0:1789760785.221752:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.221761:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.221772:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73068544, 73072640) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.221787:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73068544=0x45af000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.221845:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.221849:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73072640 00000020:00010000:2.0:1789760785.221857:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.221861:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.221865:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.221869:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.221872:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.221880:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.221887:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.221901:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760785.222018:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 73068544<>73072640 00000100:00100000:2.0:1789760785.222066:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760785.222082:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565513856:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.231794:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985147 gen 3 00000008:00010000:2.0:1789760785.231845:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760785.231854:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.231866:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.231932:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760785.231965:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565513856:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.232167:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17839, 17839] 00000080:00200000:3.0:1789760785.232184:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.232259:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.232535:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73072640 i_blocks 142744 00000020:00200000:2.0:1789760785.232540:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.232551:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.232586:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73072640, count: 4096 00000080:00200000:2.0:1789760785.232595:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73072640, bytes: 4096 00000080:00200000:2.0:1789760785.232606:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.232615:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.232617:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73072640, 73076736) 0 0 00000080:00200000:2.0:1789760785.232622:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.232639:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17840, 17840] 00020000:00200000:2.0:1789760785.232673:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73072640 chunk: [73072640, 73076736) 73076736, 4096 00020000:00200000:2.0:1789760785.232678:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.232698:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17840, 17840] 00000008:00010000:2.0:1789760785.232716:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.232721:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.232734:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73072640, 73076736) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.232747:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73072640=0x45b0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.232803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.232806:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73076736 00000020:00010000:2.0:1789760785.232814:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.232818:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.232821:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.232825:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.232828:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.232836:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.232843:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.232888:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760785.233487:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 73072640<>73076736 00000100:00100000:3.0:1789760785.233573:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565513984:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.256738:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985148 gen 3 00000008:00010000:2.0:1789760785.256790:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760785.256799:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.256811:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.256832:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760785.256860:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565513984:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760785.256962:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17840, 17840] 00000080:00200000:0.0:1789760785.256981:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760785.256987:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760785.257004:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73076736 i_blocks 142752 00000020:00200000:0.0:1789760785.257018:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760785.257029:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760785.257069:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73076736, count: 4096 00000080:00200000:0.0:1789760785.257077:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73076736, bytes: 4096 00000080:00200000:0.0:1789760785.257088:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760785.257287:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760785.257291:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73076736, 73080832) 0 0 00000080:00200000:0.0:1789760785.257296:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760785.257316:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17841, 17841] 00020000:00200000:0.0:1789760785.257326:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73076736 chunk: [73076736, 73080832) 73080832, 4096 00020000:00200000:0.0:1789760785.257331:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760785.257354:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17841, 17841] 00000008:00010000:0.0:1789760785.257374:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760785.257381:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760785.257393:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73076736, 73080832) append 0 append lockpos 0 00000080:00200000:0.0:1789760785.257408:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73076736=0x45b1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760785.257483:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760785.257487:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73080832 00000020:00010000:0.0:1789760785.257496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760785.257499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.257503:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.257506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760785.257509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760785.257516:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760785.257523:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760785.257541:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760785.257705:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 73076736<>73080832 00000100:00100000:0.0:1789760785.257747:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760785.257769:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565514112:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760785.278709:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985149 gen 3 00000008:00010000:0.0:1789760785.278762:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:0.0:1789760785.278770:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760785.278780:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760785.278864:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:0.0:1789760785.278894:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565514112:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.283474:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17841, 17841] 00000080:00200000:2.0:1789760785.283507:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.283514:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.283534:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73080832 i_blocks 142760 00000020:00200000:2.0:1789760785.283539:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.283558:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.283593:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73080832, count: 4096 00000080:00200000:2.0:1789760785.283603:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73080832, bytes: 4096 00000080:00200000:2.0:1789760785.283618:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.283627:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.283630:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73080832, 73084928) 0 0 00000080:00200000:2.0:1789760785.283635:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.283667:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17842, 17842] 00020000:00200000:2.0:1789760785.283677:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73080832 chunk: [73080832, 73084928) 73084928, 4096 00020000:00200000:2.0:1789760785.283681:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.283703:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17842, 17842] 00000008:00010000:2.0:1789760785.283721:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.283727:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.283738:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73080832, 73084928) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.283752:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73080832=0x45b2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.283822:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.283825:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73084928 00000020:00010000:2.0:1789760785.283833:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.283836:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.283840:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.283843:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.283846:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.283853:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.283859:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.283887:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760785.284290:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 73080832<>73084928 00000100:00100000:3.0:1789760785.284397:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565514240:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760785.315913:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985150 gen 3 00000008:00010000:3.0:1789760785.315965:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760785.315974:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760785.315986:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760785.316060:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760785.316093:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565514240:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760785.316408:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17842, 17842] 00000080:00200000:1.0:1789760785.316594:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.316600:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760785.316616:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73084928 i_blocks 142768 00000020:00200000:1.0:1789760785.316620:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.316733:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.316772:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73084928, count: 4096 00000080:00200000:1.0:1789760785.317549:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73084928, bytes: 4096 00000080:00200000:1.0:1789760785.317565:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.317574:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.317576:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73084928, 73089024) 0 0 00000080:00200000:1.0:1789760785.317582:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.317599:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17843, 17843] 00020000:00200000:1.0:1789760785.317608:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73084928 chunk: [73084928, 73089024) 73089024, 4096 00020000:00200000:1.0:1789760785.317612:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.320028:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17843, 17843] 00000008:00010000:1.0:1789760785.320072:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.320079:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.320094:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73084928, 73089024) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.320114:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73084928=0x45b3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.320326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.320330:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73089024 00000020:00010000:1.0:1789760785.320340:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.320344:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.320347:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.320350:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.320354:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.320364:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.320373:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.321027:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760785.321294:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73084928<>73089024 00000100:00100000:0.0:1789760785.321379:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565514368:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.342278:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985151 gen 3 00000008:00010000:1.0:1789760785.342333:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760785.342341:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.342351:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.342416:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760785.342447:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565514368:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760785.342630:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17843, 17843] 00000080:00200000:0.0:1789760785.342775:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760785.342782:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760785.342819:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73089024 i_blocks 142776 00000020:00200000:0.0:1789760785.342824:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760785.342835:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760785.342944:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73089024, count: 4096 00000080:00200000:0.0:1789760785.342953:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73089024, bytes: 4096 00000080:00200000:0.0:1789760785.342966:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760785.342974:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760785.342976:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73089024, 73093120) 0 0 00000080:00200000:0.0:1789760785.342981:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760785.342996:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17844, 17844] 00020000:00200000:0.0:1789760785.343007:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73089024 chunk: [73089024, 73093120) 73093120, 4096 00020000:00200000:0.0:1789760785.343011:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760785.343033:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17844, 17844] 00000008:00010000:0.0:1789760785.343052:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760785.343058:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760785.343069:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73089024, 73093120) append 0 append lockpos 0 00000080:00200000:0.0:1789760785.343083:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73089024=0x45b4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760785.343140:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760785.343301:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73093120 00000020:00010000:0.0:1789760785.343314:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760785.343317:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.343321:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.343324:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760785.343327:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760785.343337:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760785.343345:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760785.343362:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760785.343565:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 73089024<>73093120 00000100:00100000:1.0:1789760785.343639:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565514496:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.361694:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985152 gen 3 00000008:00010000:1.0:1789760785.361753:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760785.361762:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.361774:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.361788:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760785.361822:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565514496:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760785.362105:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17844, 17844] 00000080:00200000:0.0:1789760785.362123:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760785.362142:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760785.362159:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73093120 i_blocks 142784 00000020:00200000:0.0:1789760785.362163:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760785.362176:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760785.362211:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73093120, count: 4096 00000080:00200000:0.0:1789760785.362219:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73093120, bytes: 4096 00000080:00200000:0.0:1789760785.362229:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760785.362238:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760785.362241:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73093120, 73097216) 0 0 00000080:00200000:0.0:1789760785.362245:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760785.362265:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17845, 17845] 00020000:00200000:0.0:1789760785.362274:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73093120 chunk: [73093120, 73097216) 73097216, 4096 00020000:00200000:0.0:1789760785.362279:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760785.362300:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17845, 17845] 00000008:00010000:0.0:1789760785.362320:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760785.362327:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760785.362340:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73093120, 73097216) append 0 append lockpos 0 00000080:00200000:0.0:1789760785.362354:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73093120=0x45b5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760785.362430:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760785.362433:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73097216 00000020:00010000:0.0:1789760785.362442:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760785.362446:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.362449:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.362453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760785.362467:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760785.362476:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760785.362486:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760785.362504:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760785.362640:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42680 - object 0x240000400:5 offset 73093120<>73097216 00000100:00100000:1.0:1789760785.362825:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760785.362844:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565514624:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.386954:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985153 gen 3 00000008:00010000:1.0:1789760785.387010:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760785.387018:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.387029:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.387048:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:1.0:1789760785.387078:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565514624:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760785.387628:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17845, 17845] 00000080:00200000:0.0:1789760785.387674:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760785.387681:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760785.387697:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73097216 i_blocks 142792 00000020:00200000:0.0:1789760785.387701:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760785.387714:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760785.387748:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73097216, count: 4096 00000080:00200000:0.0:1789760785.387756:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73097216, bytes: 4096 00000080:00200000:0.0:1789760785.387766:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760785.387775:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760785.387777:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73097216, 73101312) 0 0 00000080:00200000:0.0:1789760785.387782:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760785.387798:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17846, 17846] 00020000:00200000:0.0:1789760785.387819:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73097216 chunk: [73097216, 73101312) 73101312, 4096 00020000:00200000:0.0:1789760785.387824:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760785.387846:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17846, 17846] 00000008:00010000:0.0:1789760785.387945:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760785.387952:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760785.387964:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73097216, 73101312) append 0 append lockpos 0 00000080:00200000:0.0:1789760785.387978:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73097216=0x45b6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760785.388040:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760785.388043:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73101312 00000020:00010000:0.0:1789760785.388050:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760785.388054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.388057:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.388061:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760785.388064:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760785.388072:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760785.388080:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760785.388100:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dcc0 00000008:00100000:0.0:1789760785.388936:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 73097216<>73101312 00000100:00100000:1.0:1789760785.389298:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760785.389327:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565514752:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.406126:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985154 gen 3 00000008:00010000:1.0:1789760785.406188:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760785.406293:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.406306:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.406330:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760785.406362:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565514752:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760785.406839:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17846, 17846] 00000080:00200000:0.0:1789760785.406856:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760785.406862:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760785.406879:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73101312 i_blocks 142800 00000020:00200000:0.0:1789760785.406883:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760785.406894:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760785.406930:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73101312, count: 4096 00000080:00200000:0.0:1789760785.406939:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73101312, bytes: 4096 00000080:00200000:0.0:1789760785.406949:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760785.406958:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760785.406961:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73101312, 73105408) 0 0 00000080:00200000:0.0:1789760785.406966:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760785.406986:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17847, 17847] 00020000:00200000:0.0:1789760785.406996:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73101312 chunk: [73101312, 73105408) 73105408, 4096 00020000:00200000:0.0:1789760785.407001:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760785.407023:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17847, 17847] 00000008:00010000:0.0:1789760785.407042:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760785.407048:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760785.407060:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73101312, 73105408) append 0 append lockpos 0 00000080:00200000:0.0:1789760785.407074:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73101312=0x45b7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760785.407142:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760785.407145:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73105408 00000020:00010000:0.0:1789760785.407154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760785.407157:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.407160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.407163:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760785.407166:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760785.407176:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760785.407183:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760785.407295:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760785.407424:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40700 - object 0x240000400:5 offset 73101312<>73105408 00000100:00100000:1.0:1789760785.407617:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760785.407639:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565514880:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760785.428023:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985155 gen 3 00000008:00010000:0.0:1789760785.428077:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:0.0:1789760785.428086:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760785.428097:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760785.428122:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:0.0:1789760785.428152:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565514880:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760785.428804:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17847, 17847] 00000080:00200000:1.0:1789760785.428821:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.428827:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760785.428843:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73105408 i_blocks 142808 00000020:00200000:1.0:1789760785.428847:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.428859:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.428892:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73105408, count: 4096 00000080:00200000:1.0:1789760785.428900:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73105408, bytes: 4096 00000080:00200000:1.0:1789760785.428911:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.428921:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.428924:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73105408, 73109504) 0 0 00000080:00200000:1.0:1789760785.428929:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.428947:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17848, 17848] 00020000:00200000:1.0:1789760785.428957:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73105408 chunk: [73105408, 73109504) 73109504, 4096 00020000:00200000:1.0:1789760785.428962:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.428985:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17848, 17848] 00000008:00010000:1.0:1789760785.429007:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.429013:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.429025:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73105408, 73109504) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.429039:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73105408=0x45b8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.429105:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.429108:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73109504 00000020:00010000:1.0:1789760785.429117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.429120:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.429123:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.429127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.429131:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.429139:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.429146:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.429164:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760785.429587:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73105408<>73109504 00000100:00100000:1.0:1789760785.429937:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760785.429963:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565515008:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.444885:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985156 gen 3 00000008:00010000:1.0:1789760785.444934:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760785.444944:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.444955:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.444985:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760785.445016:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565515008:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760785.445599:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17848, 17848] 00000080:00200000:0.0:1789760785.445617:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760785.445739:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760785.445758:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73109504 i_blocks 142816 00000020:00200000:0.0:1789760785.445762:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760785.445774:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760785.445880:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73109504, count: 4096 00000080:00200000:0.0:1789760785.445889:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73109504, bytes: 4096 00000080:00200000:0.0:1789760785.445903:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760785.445911:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760785.445914:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73109504, 73113600) 0 0 00000080:00200000:0.0:1789760785.445919:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760785.445936:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17849, 17849] 00020000:00200000:0.0:1789760785.445946:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73109504 chunk: [73109504, 73113600) 73113600, 4096 00020000:00200000:0.0:1789760785.445951:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760785.445974:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17849, 17849] 00000008:00010000:0.0:1789760785.445994:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760785.446000:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760785.446012:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73109504, 73113600) append 0 append lockpos 0 00000080:00200000:0.0:1789760785.446026:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73109504=0x45b9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760785.446088:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760785.446091:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73113600 00000020:00010000:0.0:1789760785.446100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760785.446104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.446107:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760785.446110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760785.446113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760785.446122:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760785.446131:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760785.446151:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760785.447638:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40700 - object 0x240000400:5 offset 73109504<>73113600 00000100:00100000:0.0:1789760785.447734:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760785.447758:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565515136:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.460792:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985157 gen 3 00000008:00010000:1.0:1789760785.460905:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760785.460913:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.460925:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.460950:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760785.460985:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565515136:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.463767:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17849, 17849] 00000080:00200000:2.0:1789760785.463795:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.463803:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.463822:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73113600 i_blocks 142824 00000020:00200000:2.0:1789760785.463829:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.463848:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.463894:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73113600, count: 4096 00000080:00200000:2.0:1789760785.463903:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73113600, bytes: 4096 00000080:00200000:2.0:1789760785.463919:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.463928:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.463932:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73113600, 73117696) 0 0 00000080:00200000:2.0:1789760785.463936:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.463952:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17850, 17850] 00020000:00200000:2.0:1789760785.463961:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73113600 chunk: [73113600, 73117696) 73117696, 4096 00020000:00200000:2.0:1789760785.463966:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.463988:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17850, 17850] 00000008:00010000:2.0:1789760785.464009:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.464016:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.464038:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73113600, 73117696) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.464054:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73113600=0x45ba000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.464124:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.464127:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73117696 00000020:00010000:2.0:1789760785.464134:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.464138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.464141:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.464144:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.464147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.464155:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.464161:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.464181:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760785.464312:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 73113600<>73117696 00000100:00100000:3.0:1789760785.464529:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565515264:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760785.476407:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985158 gen 3 00000008:00010000:3.0:1789760785.476454:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760785.476475:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760785.476486:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760785.476508:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760785.476536:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565515264:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.476781:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17850, 17850] 00000080:00200000:2.0:1789760785.476797:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.476803:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.476818:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73117696 i_blocks 142832 00000020:00200000:2.0:1789760785.476823:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.476832:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.476866:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73117696, count: 4096 00000080:00200000:2.0:1789760785.476874:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73117696, bytes: 4096 00000080:00200000:2.0:1789760785.476885:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.476894:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.476897:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73117696, 73121792) 0 0 00000080:00200000:2.0:1789760785.476901:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.476918:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17851, 17851] 00020000:00200000:2.0:1789760785.476927:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73117696 chunk: [73117696, 73121792) 73121792, 4096 00020000:00200000:2.0:1789760785.476932:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.476952:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17851, 17851] 00000008:00010000:2.0:1789760785.476970:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.476976:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.476987:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73117696, 73121792) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.477000:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73117696=0x45bb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.477066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.477070:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73121792 00000020:00010000:2.0:1789760785.477078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.477082:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.477085:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.477088:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.477091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.477098:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.477105:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.477121:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f8441893c0 00000008:00100000:2.0:1789760785.477254:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 73117696<>73121792 00000100:00100000:3.0:1789760785.477320:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760785.477349:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565515392:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760785.486787:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985159 gen 3 00000008:00010000:3.0:1789760785.486837:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f8441893c0 00000020:00200000:3.0:1789760785.486845:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760785.486868:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760785.486890:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f8441893c0 00000100:00100000:3.0:1789760785.486920:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565515392:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.487159:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17851, 17851] 00000080:00200000:2.0:1789760785.487178:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.487184:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.487201:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73121792 i_blocks 142840 00000020:00200000:2.0:1789760785.487206:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.487217:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.487253:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73121792, count: 4096 00000080:00200000:2.0:1789760785.487261:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73121792, bytes: 4096 00000080:00200000:2.0:1789760785.487272:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.487280:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.487283:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73121792, 73125888) 0 0 00000080:00200000:2.0:1789760785.487288:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.487310:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17852, 17852] 00020000:00200000:2.0:1789760785.487320:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73121792 chunk: [73121792, 73125888) 73125888, 4096 00020000:00200000:2.0:1789760785.487324:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.487345:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17852, 17852] 00000008:00010000:2.0:1789760785.487365:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.487371:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.487383:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73121792, 73125888) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.487397:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73121792=0x45bc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.487470:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.487473:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73125888 00000020:00010000:2.0:1789760785.487482:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.487486:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.487489:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.487493:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.487496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.487504:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.487515:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.487533:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760785.488094:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 73121792<>73125888 00000100:00100000:2.0:1789760785.488140:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760785.488159:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565515520:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.495525:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985160 gen 3 00000008:00010000:2.0:1789760785.495786:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760785.495795:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.495806:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.495828:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000080:00200000:3.0:1789760785.495888:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17852, 17852] 00000080:00200000:3.0:1789760785.495908:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.495914:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.495930:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73125888 i_blocks 142848 00000020:00200000:3.0:1789760785.495933:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.495944:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.496008:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73125888, count: 4096 00000080:00200000:3.0:1789760785.496017:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73125888, bytes: 4096 00000080:00200000:3.0:1789760785.496030:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.496039:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.496041:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73125888, 73129984) 0 0 00000080:00200000:3.0:1789760785.496045:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.496063:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17853, 17853] 00020000:00200000:3.0:1789760785.496072:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73125888 chunk: [73125888, 73129984) 73129984, 4096 00020000:00200000:3.0:1789760785.496076:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.496098:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17853, 17853] 00000008:00010000:3.0:1789760785.496117:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.496123:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.496135:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73125888, 73129984) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.496148:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73125888=0x45bd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.496203:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.496206:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73129984 00000020:00010000:3.0:1789760785.496214:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.496218:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.496221:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.496225:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.496228:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.496236:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.496244:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:2.0:1789760785.496766:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565515520:192.168.203.147@tcp:4:dd.0 00000008:00010000:2.0:1789760785.497024:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:2.0:1789760785.497150:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 73125888<>73129984 00000100:00100000:2.0:1789760785.497216:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565515648:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.509509:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985161 gen 3 00000008:00010000:2.0:1789760785.509565:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760785.509574:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.509587:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.509610:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760785.509686:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565515648:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.509974:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17853, 17853] 00000080:00200000:3.0:1789760785.509993:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.509999:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.510015:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73129984 i_blocks 142856 00000020:00200000:3.0:1789760785.510021:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.510031:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.510066:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73129984, count: 4096 00000080:00200000:3.0:1789760785.510073:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73129984, bytes: 4096 00000080:00200000:3.0:1789760785.510085:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.510093:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.510095:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73129984, 73134080) 0 0 00000080:00200000:3.0:1789760785.510100:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.510117:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17854, 17854] 00020000:00200000:3.0:1789760785.510127:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73129984 chunk: [73129984, 73134080) 73134080, 4096 00020000:00200000:3.0:1789760785.510132:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.510152:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17854, 17854] 00000008:00010000:3.0:1789760785.510170:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.510176:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.510187:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73129984, 73134080) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.510202:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73129984=0x45be000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.510260:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.510264:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73134080 00000020:00010000:3.0:1789760785.510272:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.510276:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.510279:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.510283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.510286:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.510294:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.510301:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.510318:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760785.510469:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 73129984<>73134080 00000100:00100000:2.0:1789760785.510720:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565515776:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.521560:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985162 gen 3 00000008:00010000:2.0:1789760785.521612:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760785.521621:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.521633:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.521693:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760785.521726:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565515776:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.522260:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17854, 17854] 00000080:00200000:3.0:1789760785.522278:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.522284:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.522302:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73134080 i_blocks 142864 00000020:00200000:3.0:1789760785.522306:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.522317:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.522353:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73134080, count: 4096 00000080:00200000:3.0:1789760785.522360:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73134080, bytes: 4096 00000080:00200000:3.0:1789760785.522373:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.522382:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.522384:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73134080, 73138176) 0 0 00000080:00200000:3.0:1789760785.522389:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.522483:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17855, 17855] 00020000:00200000:3.0:1789760785.522493:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73134080 chunk: [73134080, 73138176) 73138176, 4096 00020000:00200000:3.0:1789760785.522498:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.522524:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17855, 17855] 00000008:00010000:3.0:1789760785.522543:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.522550:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.522563:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73134080, 73138176) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.522577:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73134080=0x45bf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.522638:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.523984:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73138176 00000020:00010000:3.0:1789760785.524010:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.524015:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.524019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.524024:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.524027:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.524042:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.524053:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.524092:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760785.524254:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 73134080<>73138176 00000100:00100000:2.0:1789760785.524449:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760785.524489:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565515904:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.533982:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985163 gen 3 00000008:00010000:2.0:1789760785.534037:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760785.534047:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.534061:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.534085:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760785.534118:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565515904:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.535138:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17855, 17855] 00000080:00200000:3.0:1789760785.535167:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.535174:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.535851:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73138176 i_blocks 142872 00000020:00200000:3.0:1789760785.535861:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.535876:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.535912:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73138176, count: 4096 00000080:00200000:3.0:1789760785.535920:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73138176, bytes: 4096 00000080:00200000:3.0:1789760785.535935:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.535945:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.535947:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73138176, 73142272) 0 0 00000080:00200000:3.0:1789760785.535952:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.535970:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17856, 17856] 00020000:00200000:3.0:1789760785.535980:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73138176 chunk: [73138176, 73142272) 73142272, 4096 00020000:00200000:3.0:1789760785.535984:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.536007:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17856, 17856] 00000008:00010000:3.0:1789760785.536026:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.536033:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.536046:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73138176, 73142272) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.536061:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73138176=0x45c0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.536133:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.536136:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73142272 00000020:00010000:3.0:1789760785.536144:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.536148:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.536151:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.536154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.536157:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.536167:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.536174:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.536210:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760785.536830:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 73138176<>73142272 00000100:00100000:3.0:1789760785.537413:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565516032:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760785.546029:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985164 gen 3 00000008:00010000:3.0:1789760785.546076:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:3.0:1789760785.546085:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760785.546096:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760785.546117:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:3.0:1789760785.546156:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565516032:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760785.546870:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17856, 17856] 00000080:00200000:1.0:1789760785.546890:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.546895:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760785.546911:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73142272 i_blocks 142880 00000020:00200000:1.0:1789760785.546915:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.546925:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.546972:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73142272, count: 4096 00000080:00200000:1.0:1789760785.546981:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73142272, bytes: 4096 00000080:00200000:1.0:1789760785.546992:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.547002:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.547005:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73142272, 73146368) 0 0 00000080:00200000:1.0:1789760785.547010:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.547027:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17857, 17857] 00020000:00200000:1.0:1789760785.547035:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73142272 chunk: [73142272, 73146368) 73146368, 4096 00020000:00200000:1.0:1789760785.547040:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.547060:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17857, 17857] 00000008:00010000:1.0:1789760785.547076:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.547082:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.547094:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73142272, 73146368) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.547108:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73142272=0x45c1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.547169:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.547172:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73146368 00000020:00010000:1.0:1789760785.547180:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.547183:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.547198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.547201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.547204:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.547212:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.547219:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.547238:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760785.547372:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73142272<>73146368 00000100:00100000:1.0:1789760785.547428:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760785.547447:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565516160:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.558023:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985165 gen 3 00000008:00010000:1.0:1789760785.558074:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760785.558083:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.558095:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.558123:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760785.558155:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565516160:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.558268:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17857, 17857] 00000080:00200000:3.0:1789760785.558284:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.558291:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.558306:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73146368 i_blocks 142888 00000020:00200000:3.0:1789760785.558309:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.558320:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.558355:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73146368, count: 4096 00000080:00200000:3.0:1789760785.558362:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73146368, bytes: 4096 00000080:00200000:3.0:1789760785.558373:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.558381:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.558383:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73146368, 73150464) 0 0 00000080:00200000:3.0:1789760785.558388:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.558418:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17858, 17858] 00020000:00200000:3.0:1789760785.558427:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73146368 chunk: [73146368, 73150464) 73150464, 4096 00020000:00200000:3.0:1789760785.558432:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.558463:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17858, 17858] 00000008:00010000:3.0:1789760785.558483:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.558496:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.558510:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73146368, 73150464) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.558525:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73146368=0x45c2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.558590:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.558593:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73150464 00000020:00010000:3.0:1789760785.558602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.558606:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.558610:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.558614:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.558617:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.558625:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.558633:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.558686:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760785.558826:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 73146368<>73150464 00000100:00100000:3.0:1789760785.558869:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760785.558885:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565516288:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760785.569205:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985166 gen 3 00000008:00010000:3.0:1789760785.569267:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:3.0:1789760785.569275:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760785.569287:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760785.569313:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:3.0:1789760785.569342:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565516288:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760785.569467:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17858, 17858] 00000080:00200000:1.0:1789760785.569492:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.569498:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760785.569513:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73150464 i_blocks 142896 00000020:00200000:1.0:1789760785.569517:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.569527:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.569561:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73150464, count: 4096 00000080:00200000:1.0:1789760785.569568:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73150464, bytes: 4096 00000080:00200000:1.0:1789760785.569578:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.569586:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.569589:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73150464, 73154560) 0 0 00000080:00200000:1.0:1789760785.569594:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.569611:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17859, 17859] 00020000:00200000:1.0:1789760785.569620:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73150464 chunk: [73150464, 73154560) 73154560, 4096 00020000:00200000:1.0:1789760785.569625:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.569667:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17859, 17859] 00000008:00010000:1.0:1789760785.569686:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.569691:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.569703:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73150464, 73154560) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.569716:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73150464=0x45c3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.569774:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.569777:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73154560 00000020:00010000:1.0:1789760785.569784:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.569788:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.569790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.569794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.569796:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.569815:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.569822:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.569839:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760785.569971:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73150464<>73154560 00000100:00100000:0.0:1789760785.570150:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760785.570174:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565516416:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.578920:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985167 gen 3 00000008:00010000:1.0:1789760785.578972:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760785.578981:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.578992:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.579015:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760785.579044:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565516416:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.579372:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17859, 17859] 00000080:00200000:3.0:1789760785.579389:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.579395:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.579420:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73154560 i_blocks 142904 00000020:00200000:3.0:1789760785.579424:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.579434:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.579474:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73154560, count: 4096 00000080:00200000:3.0:1789760785.579482:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73154560, bytes: 4096 00000080:00200000:3.0:1789760785.579492:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.579501:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.579503:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73154560, 73158656) 0 0 00000080:00200000:3.0:1789760785.579508:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.579525:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17860, 17860] 00020000:00200000:3.0:1789760785.579533:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73154560 chunk: [73154560, 73158656) 73158656, 4096 00020000:00200000:3.0:1789760785.579538:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.579557:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17860, 17860] 00000008:00010000:3.0:1789760785.579574:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.579580:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.579591:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73154560, 73158656) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.579605:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73154560=0x45c4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.579695:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.579698:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73158656 00000020:00010000:3.0:1789760785.579707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.579711:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.579714:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.579717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.579721:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.579730:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.579736:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.579753:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760785.579883:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 73154560<>73158656 00000100:00100000:2.0:1789760785.580141:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565516544:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.594094:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985168 gen 3 00000008:00010000:2.0:1789760785.594143:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760785.594152:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.594163:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.594185:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760785.594216:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565516544:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.594702:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17860, 17860] 00000080:00200000:3.0:1789760785.594719:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.594726:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.594743:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73158656 i_blocks 142912 00000020:00200000:3.0:1789760785.594747:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.594758:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.594791:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73158656, count: 4096 00000080:00200000:3.0:1789760785.594811:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73158656, bytes: 4096 00000080:00200000:3.0:1789760785.594822:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.594832:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.594835:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73158656, 73162752) 0 0 00000080:00200000:3.0:1789760785.594839:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.594857:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17861, 17861] 00020000:00200000:3.0:1789760785.594865:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73158656 chunk: [73158656, 73162752) 73162752, 4096 00020000:00200000:3.0:1789760785.594870:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.594889:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17861, 17861] 00000008:00010000:3.0:1789760785.594909:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.594915:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.594927:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73158656, 73162752) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.594942:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73158656=0x45c5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.595006:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.595009:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73162752 00000020:00010000:3.0:1789760785.595018:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.595021:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.595024:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.595028:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.595031:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.595040:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.595048:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.595065:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760785.595194:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 73158656<>73162752 00000100:00100000:2.0:1789760785.595531:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565516672:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.603599:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985169 gen 3 00000008:00010000:2.0:1789760785.603695:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760785.603704:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.603717:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.603740:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760785.603770:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565516672:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.604149:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17861, 17861] 00000080:00200000:3.0:1789760785.604165:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.604171:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.604189:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73162752 i_blocks 142920 00000020:00200000:3.0:1789760785.604193:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.604204:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.604294:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73162752, count: 4096 00000080:00200000:3.0:1789760785.604312:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73162752, bytes: 4096 00000080:00200000:3.0:1789760785.604323:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.604333:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.604335:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73162752, 73166848) 0 0 00000080:00200000:3.0:1789760785.604340:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.604357:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17862, 17862] 00020000:00200000:3.0:1789760785.604366:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73162752 chunk: [73162752, 73166848) 73166848, 4096 00020000:00200000:3.0:1789760785.604371:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.604391:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17862, 17862] 00000008:00010000:3.0:1789760785.604409:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.604415:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.604429:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73162752, 73166848) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.604443:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73162752=0x45c6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.604521:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.604524:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73166848 00000020:00010000:3.0:1789760785.604532:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.604545:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.604549:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.604553:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.604556:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.604565:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.604572:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.604591:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760785.604767:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 73162752<>73166848 00000100:00100000:2.0:1789760785.604896:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565516800:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.633583:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985170 gen 3 00000008:00010000:2.0:1789760785.633692:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760785.633701:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.633714:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.633871:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760785.633894:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565516800:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.636193:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17862, 17862] 00000080:00200000:3.0:1789760785.636224:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.636232:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.636262:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73166848 i_blocks 142928 00000020:00200000:3.0:1789760785.636269:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.636285:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.636322:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73166848, count: 4096 00000080:00200000:3.0:1789760785.636332:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73166848, bytes: 4096 00000080:00200000:3.0:1789760785.636348:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.636358:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.636360:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73166848, 73170944) 0 0 00000080:00200000:3.0:1789760785.636364:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.636381:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17863, 17863] 00020000:00200000:3.0:1789760785.636390:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73166848 chunk: [73166848, 73170944) 73170944, 4096 00020000:00200000:3.0:1789760785.636407:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.636430:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17863, 17863] 00000008:00010000:3.0:1789760785.636449:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.636463:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.636476:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73166848, 73170944) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.636490:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73166848=0x45c7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.636569:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.636573:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73170944 00000020:00010000:3.0:1789760785.636581:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.636585:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.636587:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.636591:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.636594:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.636603:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.636610:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.636633:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760785.636826:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 73166848<>73170944 00000100:00100000:2.0:1789760785.637117:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565516928:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.655742:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985171 gen 3 00000008:00010000:2.0:1789760785.655802:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760785.655811:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.655823:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.655847:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760785.655877:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565516928:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.656204:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17863, 17863] 00000080:00200000:3.0:1789760785.656222:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.656229:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.656245:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73170944 i_blocks 142936 00000020:00200000:3.0:1789760785.656249:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.656259:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.656295:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73170944, count: 4096 00000080:00200000:3.0:1789760785.656303:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73170944, bytes: 4096 00000080:00200000:3.0:1789760785.656313:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.656321:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.656324:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73170944, 73175040) 0 0 00000080:00200000:3.0:1789760785.656328:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.656344:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17864, 17864] 00020000:00200000:3.0:1789760785.656352:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73170944 chunk: [73170944, 73175040) 73175040, 4096 00020000:00200000:3.0:1789760785.656357:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.656377:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17864, 17864] 00000008:00010000:3.0:1789760785.656396:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.656402:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.656415:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73170944, 73175040) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.656428:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73170944=0x45c8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.656507:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.656510:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73175040 00000020:00010000:3.0:1789760785.656519:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.656527:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.656530:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.656534:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.656546:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.656563:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.656570:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.656587:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760785.656747:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 73170944<>73175040 00000100:00100000:2.0:1789760785.659506:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565517056:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.680136:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985172 gen 3 00000008:00010000:2.0:1789760785.680185:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760785.680194:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.680206:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.680239:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760785.680270:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565517056:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.680782:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17864, 17864] 00000080:00200000:3.0:1789760785.680800:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.680806:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.680823:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73175040 i_blocks 142944 00000020:00200000:3.0:1789760785.680827:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.680837:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.680870:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73175040, count: 4096 00000080:00200000:3.0:1789760785.680877:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73175040, bytes: 4096 00000080:00200000:3.0:1789760785.680890:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.680899:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.680901:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73175040, 73179136) 0 0 00000080:00200000:3.0:1789760785.680905:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.680922:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17865, 17865] 00020000:00200000:3.0:1789760785.680931:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73175040 chunk: [73175040, 73179136) 73179136, 4096 00020000:00200000:3.0:1789760785.680936:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.680955:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17865, 17865] 00000008:00010000:3.0:1789760785.680973:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.680979:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.680991:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73175040, 73179136) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.681005:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73175040=0x45c9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.681078:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.681081:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73179136 00000020:00010000:3.0:1789760785.681089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.681093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.681095:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.681099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.681102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.681111:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.681117:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.681134:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760785.681363:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 73175040<>73179136 00000100:00100000:2.0:1789760785.682947:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565517184:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.700355:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985173 gen 3 00000008:00010000:2.0:1789760785.700406:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760785.700413:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.700425:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.700631:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760785.700715:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565517184:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.705921:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17865, 17865] 00000080:00200000:3.0:1789760785.705950:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.705957:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.705976:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73179136 i_blocks 142952 00000020:00200000:3.0:1789760785.705981:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.705998:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.706033:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73179136, count: 4096 00000080:00200000:3.0:1789760785.706043:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73179136, bytes: 4096 00000080:00200000:3.0:1789760785.706059:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.706069:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.706071:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73179136, 73183232) 0 0 00000080:00200000:3.0:1789760785.706076:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.706094:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17866, 17866] 00020000:00200000:3.0:1789760785.706103:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73179136 chunk: [73179136, 73183232) 73183232, 4096 00020000:00200000:3.0:1789760785.706108:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.706131:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17866, 17866] 00000008:00010000:3.0:1789760785.706152:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.706158:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.706171:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73179136, 73183232) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.706188:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73179136=0x45ca000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.706271:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.706274:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73183232 00000020:00010000:3.0:1789760785.706282:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.706286:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.706289:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.706293:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.706296:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.706306:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.706314:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.706343:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760785.706481:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66d80 - object 0x240000400:5 offset 73179136<>73183232 00000100:00100000:2.0:1789760785.706765:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760785.706793:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565517312:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.726944:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985174 gen 3 00000008:00010000:2.0:1789760785.726996:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760785.727004:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.727017:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.727042:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760785.727071:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565517312:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.727849:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17866, 17866] 00000080:00200000:3.0:1789760785.727868:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.727876:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.727894:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73183232 i_blocks 142960 00000020:00200000:3.0:1789760785.727899:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.727911:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.727942:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73183232, count: 4096 00000080:00200000:3.0:1789760785.727951:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73183232, bytes: 4096 00000080:00200000:3.0:1789760785.727962:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.727970:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.727973:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73183232, 73187328) 0 0 00000080:00200000:3.0:1789760785.727978:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.727996:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17867, 17867] 00020000:00200000:3.0:1789760785.728005:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73183232 chunk: [73183232, 73187328) 73187328, 4096 00020000:00200000:3.0:1789760785.728009:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.728031:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17867, 17867] 00000008:00010000:3.0:1789760785.728049:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.728055:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.728068:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73183232, 73187328) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.728081:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73183232=0x45cb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.728143:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.728146:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73187328 00000020:00010000:3.0:1789760785.728155:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.728159:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.728161:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.728165:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.728168:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.728177:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.728185:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.728203:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760785.728490:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 73183232<>73187328 00000100:00100000:2.0:1789760785.728790:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565517440:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.744176:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985175 gen 3 00000008:00010000:2.0:1789760785.746166:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760785.746191:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.746205:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.746897:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:2.0:1789760785.746935:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565517440:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.751498:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17867, 17867] 00000080:00200000:3.0:1789760785.751538:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.751546:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.751565:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73187328 i_blocks 142968 00000020:00200000:3.0:1789760785.751571:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.751586:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.751619:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73187328, count: 4096 00000080:00200000:3.0:1789760785.751629:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73187328, bytes: 4096 00000080:00200000:3.0:1789760785.751661:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.751672:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.751674:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73187328, 73191424) 0 0 00000080:00200000:3.0:1789760785.751679:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.751698:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17868, 17868] 00020000:00200000:3.0:1789760785.751707:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73187328 chunk: [73187328, 73191424) 73191424, 4096 00020000:00200000:3.0:1789760785.751713:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.751735:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17868, 17868] 00000008:00010000:3.0:1789760785.751754:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.751760:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.751772:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73187328, 73191424) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.751787:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73187328=0x45cc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.751863:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.751866:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73191424 00000020:00010000:3.0:1789760785.751876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.751880:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.751883:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.751886:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.751889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.751898:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.751906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.751932:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760785.752441:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 73187328<>73191424 00000100:00100000:2.0:1789760785.752727:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565517568:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.769343:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985176 gen 3 00000008:00010000:2.0:1789760785.769395:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760785.769404:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.769416:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.769691:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760785.769723:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565517568:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.769948:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17868, 17868] 00000080:00200000:3.0:1789760785.769966:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.769973:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.770057:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73191424 i_blocks 142976 00000020:00200000:2.0:1789760785.770062:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.770073:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.770109:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73191424, count: 4096 00000080:00200000:2.0:1789760785.770122:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73191424, bytes: 4096 00000080:00200000:2.0:1789760785.770132:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.770142:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.770145:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73191424, 73195520) 0 0 00000080:00200000:2.0:1789760785.770149:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.770167:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17869, 17869] 00020000:00200000:2.0:1789760785.770177:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73191424 chunk: [73191424, 73195520) 73195520, 4096 00020000:00200000:2.0:1789760785.770182:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.770202:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17869, 17869] 00000008:00010000:2.0:1789760785.770221:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.770226:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.770237:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73191424, 73195520) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.770252:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73191424=0x45cd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.770305:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.770308:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73195520 00000020:00010000:2.0:1789760785.770317:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.770320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.770323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.770327:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.770330:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.770337:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.770344:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.770403:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760785.770546:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 73191424<>73195520 00000100:00100000:2.0:1789760785.770592:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760785.770609:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565517696:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.789214:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985177 gen 3 00000008:00010000:2.0:1789760785.789265:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760785.789273:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.789283:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.789355:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:2.0:1789760785.789387:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565517696:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.796533:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17869, 17869] 00000080:00200000:3.0:1789760785.796561:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.796568:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.796843:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73195520 i_blocks 142984 00000020:00200000:2.0:1789760785.796919:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.796937:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.796974:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73195520, count: 4096 00000080:00200000:2.0:1789760785.796983:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73195520, bytes: 4096 00000080:00200000:2.0:1789760785.796998:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.797007:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.797010:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73195520, 73199616) 0 0 00000080:00200000:2.0:1789760785.797015:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.797037:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17870, 17870] 00020000:00200000:2.0:1789760785.797047:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73195520 chunk: [73195520, 73199616) 73199616, 4096 00020000:00200000:2.0:1789760785.797052:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.797075:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17870, 17870] 00000008:00010000:2.0:1789760785.797096:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.797103:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.797114:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73195520, 73199616) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.797129:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73195520=0x45ce000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.797195:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.797198:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73199616 00000020:00010000:2.0:1789760785.797207:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.797210:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.797213:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.797217:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.797219:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.797228:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.797235:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.797422:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760785.797554:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 73195520<>73199616 00000100:00100000:3.0:1789760785.797683:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565517824:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.808926:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985178 gen 3 00000008:00010000:2.0:1789760785.808975:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760785.808983:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.808995:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.809029:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:2.0:1789760785.809060:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565517824:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760785.809171:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17870, 17870] 00000080:00200000:1.0:1789760785.809188:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.809193:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760785.809209:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73199616 i_blocks 142992 00000020:00200000:1.0:1789760785.809213:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.809224:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.809271:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73199616, count: 4096 00000080:00200000:1.0:1789760785.809279:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73199616, bytes: 4096 00000080:00200000:1.0:1789760785.809289:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.809297:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.809300:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73199616, 73203712) 0 0 00000080:00200000:1.0:1789760785.809304:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.809321:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17871, 17871] 00020000:00200000:1.0:1789760785.809330:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73199616 chunk: [73199616, 73203712) 73203712, 4096 00020000:00200000:1.0:1789760785.809334:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.809354:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17871, 17871] 00000008:00010000:1.0:1789760785.809372:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.809379:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.809391:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73199616, 73203712) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.809405:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73199616=0x45cf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.809483:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.809486:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73203712 00000020:00010000:1.0:1789760785.809494:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.809499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.809502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.809506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.809509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.809517:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.809523:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.809540:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760785.809713:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73199616<>73203712 00000100:00100000:0.0:1789760785.810007:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760785.810028:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565517952:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.820156:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985179 gen 3 00000008:00010000:1.0:1789760785.820206:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760785.820215:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.820227:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.820254:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760785.820284:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565517952:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.820723:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17871, 17871] 00000080:00200000:2.0:1789760785.820740:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.820746:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.820762:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73203712 i_blocks 143000 00000020:00200000:2.0:1789760785.820768:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.820779:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.820813:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73203712, count: 4096 00000080:00200000:2.0:1789760785.820820:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73203712, bytes: 4096 00000080:00200000:2.0:1789760785.820832:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.820841:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.820843:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73203712, 73207808) 0 0 00000080:00200000:2.0:1789760785.820848:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.820866:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17872, 17872] 00020000:00200000:2.0:1789760785.820875:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73203712 chunk: [73203712, 73207808) 73207808, 4096 00020000:00200000:2.0:1789760785.820880:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.820899:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17872, 17872] 00000008:00010000:2.0:1789760785.820915:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.820920:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.820932:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73203712, 73207808) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.820945:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73203712=0x45d0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.821002:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.821226:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73207808 00000020:00010000:2.0:1789760785.821238:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.821242:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.821245:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.821249:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.821252:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.821261:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.821268:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.821289:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760785.821407:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 73203712<>73207808 00000100:00100000:3.0:1789760785.821500:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565518080:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.837713:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985180 gen 3 00000008:00010000:2.0:1789760785.837764:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760785.837773:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.837783:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.837866:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17872, 17872] 00000008:00010000:2.0:1789760785.837869:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000080:00200000:1.0:1789760785.837886:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.837893:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:2.0:1789760785.837901:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565518080:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760785.837909:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73207808 i_blocks 143008 00000020:00200000:1.0:1789760785.837915:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.837928:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.837963:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73207808, count: 4096 00000080:00200000:1.0:1789760785.837972:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73207808, bytes: 4096 00000080:00200000:1.0:1789760785.837982:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.837989:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.837991:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73207808, 73211904) 0 0 00000080:00200000:1.0:1789760785.837996:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.838022:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17873, 17873] 00020000:00200000:1.0:1789760785.838030:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73207808 chunk: [73207808, 73211904) 73211904, 4096 00020000:00200000:1.0:1789760785.838033:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.838050:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17873, 17873] 00000008:00010000:1.0:1789760785.838065:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.838070:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.838080:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73207808, 73211904) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.838091:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73207808=0x45d1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.838736:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.838741:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73211904 00000020:00010000:1.0:1789760785.838753:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.838758:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.838762:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.838766:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.838769:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.838778:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.838999:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.839036:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760785.839179:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73207808<>73211904 00000100:00100000:0.0:1789760785.839323:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565518208:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.848281:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985181 gen 3 00000008:00010000:1.0:1789760785.848325:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760785.848333:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.848343:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.848366:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760785.848408:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565518208:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.848884:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17873, 17873] 00000080:00200000:2.0:1789760785.848902:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.848908:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.848923:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73211904 i_blocks 143016 00000020:00200000:2.0:1789760785.848927:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.848937:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.848971:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73211904, count: 4096 00000080:00200000:2.0:1789760785.848978:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73211904, bytes: 4096 00000080:00200000:2.0:1789760785.848990:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.848999:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.849002:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73211904, 73216000) 0 0 00000080:00200000:2.0:1789760785.849006:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.849024:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17874, 17874] 00020000:00200000:2.0:1789760785.849034:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73211904 chunk: [73211904, 73216000) 73216000, 4096 00020000:00200000:2.0:1789760785.849038:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.849062:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17874, 17874] 00000008:00010000:2.0:1789760785.849083:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.849089:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.849101:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73211904, 73216000) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.849115:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73211904=0x45d2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.849173:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.849176:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73216000 00000020:00010000:2.0:1789760785.849184:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.849187:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.849190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.849194:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.849196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.849204:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.849211:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.849230:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760785.849359:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 73211904<>73216000 00000100:00100000:3.0:1789760785.849618:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565518336:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760785.869095:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985182 gen 3 00000008:00010000:3.0:1789760785.869146:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760785.869154:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760785.869166:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760785.869192:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760785.869226:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565518336:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.870195:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17874, 17874] 00000080:00200000:2.0:1789760785.870215:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.870221:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.870237:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73216000 i_blocks 143024 00000020:00200000:2.0:1789760785.870241:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.870253:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.870292:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73216000, count: 4096 00000080:00200000:2.0:1789760785.870299:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73216000, bytes: 4096 00000080:00200000:2.0:1789760785.870311:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.870321:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.870324:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73216000, 73220096) 0 0 00000080:00200000:2.0:1789760785.870328:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.870346:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17875, 17875] 00020000:00200000:2.0:1789760785.870356:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73216000 chunk: [73216000, 73220096) 73220096, 4096 00020000:00200000:2.0:1789760785.870361:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.870382:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17875, 17875] 00000008:00010000:2.0:1789760785.870400:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.870411:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.870422:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73216000, 73220096) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.870437:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73216000=0x45d3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.870636:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.870640:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73220096 00000020:00010000:2.0:1789760785.870700:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.870705:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.870708:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.870713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.870715:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.870725:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.870733:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.870756:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760785.871099:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 73216000<>73220096 00000100:00100000:3.0:1789760785.871164:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760785.871899:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565518464:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.880867:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985183 gen 3 00000008:00010000:2.0:1789760785.880920:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760785.880928:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.880940:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.881001:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:2.0:1789760785.881031:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565518464:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.881856:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17875, 17875] 00000080:00200000:3.0:1789760785.881905:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.881911:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.882029:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73220096 i_blocks 143032 00000020:00200000:2.0:1789760785.882034:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.882043:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.882317:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73220096, count: 4096 00000080:00200000:2.0:1789760785.882326:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73220096, bytes: 4096 00000080:00200000:2.0:1789760785.882338:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.882347:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.882350:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73220096, 73224192) 0 0 00000080:00200000:2.0:1789760785.882355:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.882372:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17876, 17876] 00020000:00200000:2.0:1789760785.882381:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73220096 chunk: [73220096, 73224192) 73224192, 4096 00020000:00200000:2.0:1789760785.882386:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.882405:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17876, 17876] 00000008:00010000:2.0:1789760785.882482:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.882627:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.882662:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73220096, 73224192) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.882679:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73220096=0x45d4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.882740:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.882743:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73224192 00000020:00010000:2.0:1789760785.882753:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.882757:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.882759:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.882763:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.882766:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.882830:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.882838:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.882887:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760785.883020:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 73220096<>73224192 00000100:00100000:3.0:1789760785.883106:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565518592:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.893364:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985184 gen 3 00000008:00010000:2.0:1789760785.893423:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760785.893433:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.893444:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.893480:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000080:00200000:1.0:1789760785.893575:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17876, 17876] 00000080:00200000:1.0:1789760785.893593:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.893600:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760785.893615:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73224192 i_blocks 143040 00000020:00200000:1.0:1789760785.893618:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.893629:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.893698:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73224192, count: 4096 00000080:00200000:1.0:1789760785.893707:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73224192, bytes: 4096 00000080:00200000:1.0:1789760785.893728:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.893739:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.893741:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73224192, 73228288) 0 0 00000080:00200000:1.0:1789760785.893746:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.893765:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17877, 17877] 00020000:00200000:1.0:1789760785.893773:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73224192 chunk: [73224192, 73228288) 73228288, 4096 00020000:00200000:1.0:1789760785.893778:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.893798:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17877, 17877] 00000008:00010000:1.0:1789760785.893816:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.893822:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.893835:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73224192, 73228288) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.893850:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73224192=0x45d5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.893914:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.893917:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73228288 00000020:00010000:1.0:1789760785.893925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.893929:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.893931:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.893935:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.893937:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.893945:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.893952:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.893974:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760785.894096:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73224192<>73228288 00000100:00100000:1.0:1789760785.894149:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760785.894174:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565518720:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.894678:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565518592:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760785.905135:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985185 gen 3 00000008:00010000:1.0:1789760785.905184:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760785.905192:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760785.905203:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760785.905228:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760785.905443:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565518720:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.905895:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17877, 17877] 00000080:00200000:2.0:1789760785.905914:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.905921:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.905937:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73228288 i_blocks 143048 00000020:00200000:2.0:1789760785.905941:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.905952:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.905986:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73228288, count: 4096 00000080:00200000:2.0:1789760785.906874:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73228288, bytes: 4096 00000080:00200000:2.0:1789760785.906889:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.906900:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.906903:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73228288, 73232384) 0 0 00000080:00200000:2.0:1789760785.906908:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.906983:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17878, 17878] 00020000:00200000:2.0:1789760785.906995:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73228288 chunk: [73228288, 73232384) 73232384, 4096 00020000:00200000:2.0:1789760785.906999:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.907022:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17878, 17878] 00000008:00010000:2.0:1789760785.907042:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.907049:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.907061:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73228288, 73232384) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.907075:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73228288=0x45d6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.907138:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.907141:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73232384 00000020:00010000:2.0:1789760785.907150:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.907153:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.907156:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.907159:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.907162:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.907171:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.907297:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.907321:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760785.907450:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 73228288<>73232384 00000100:00100000:3.0:1789760785.907669:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565518848:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760785.919039:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985186 gen 3 00000008:00010000:3.0:1789760785.919090:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760785.919098:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760785.919109:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760785.919134:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760785.919163:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565518848:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.920074:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17878, 17878] 00000080:00200000:2.0:1789760785.920094:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.920100:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.920117:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73232384 i_blocks 143056 00000020:00200000:2.0:1789760785.920121:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.920132:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.920166:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73232384, count: 4096 00000080:00200000:2.0:1789760785.920183:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73232384, bytes: 4096 00000080:00200000:2.0:1789760785.920196:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.920205:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.920207:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73232384, 73236480) 0 0 00000080:00200000:2.0:1789760785.921423:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.921451:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17879, 17879] 00020000:00200000:2.0:1789760785.921475:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73232384 chunk: [73232384, 73236480) 73236480, 4096 00020000:00200000:2.0:1789760785.921480:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.921505:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17879, 17879] 00000008:00010000:2.0:1789760785.921527:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.921533:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.921546:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73232384, 73236480) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.921563:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73232384=0x45d7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.921630:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.921634:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73236480 00000020:00010000:2.0:1789760785.921659:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.921663:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.921667:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.921671:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.921674:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.921683:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.921690:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.921714:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760785.921833:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 73232384<>73236480 00000100:00100000:3.0:1789760785.922101:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760785.922126:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565518976:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760785.935081:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985187 gen 3 00000008:00010000:3.0:1789760785.935130:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760785.935138:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760785.935150:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760785.935180:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760785.935212:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565518976:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.935516:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17879, 17879] 00000080:00200000:2.0:1789760785.935533:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.935540:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.935556:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73236480 i_blocks 143064 00000020:00200000:2.0:1789760785.935560:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.935570:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.935603:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73236480, count: 4096 00000080:00200000:2.0:1789760785.935610:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73236480, bytes: 4096 00000080:00200000:2.0:1789760785.935621:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.935630:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.935633:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73236480, 73240576) 0 0 00000080:00200000:2.0:1789760785.935638:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.936149:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17880, 17880] 00020000:00200000:2.0:1789760785.936160:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73236480 chunk: [73236480, 73240576) 73240576, 4096 00020000:00200000:2.0:1789760785.936165:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.936186:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17880, 17880] 00000008:00010000:2.0:1789760785.937731:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.937743:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.937760:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73236480, 73240576) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.937779:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73236480=0x45d8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.937852:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.937856:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73240576 00000020:00010000:2.0:1789760785.937865:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.937868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.937872:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.937876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.937878:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.937889:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.937898:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.938023:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760785.938148:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 73236480<>73240576 00000100:00100000:3.0:1789760785.938270:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565519104:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760785.946903:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760785.948504:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985188 gen 3 00000008:00010000:3.0:1789760785.948552:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760785.948561:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760785.948573:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760785.948600:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760785.948629:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565519104:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760785.949196:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17880, 17880] 00000080:00200000:1.0:1789760785.949213:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.949219:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760785.949242:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73240576 i_blocks 143072 00000020:00200000:1.0:1789760785.949246:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.949256:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.949298:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73240576, count: 4096 00000080:00200000:1.0:1789760785.949306:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73240576, bytes: 4096 00000080:00200000:1.0:1789760785.949318:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.949327:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.949330:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73240576, 73244672) 0 0 00000080:00200000:1.0:1789760785.949334:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.949351:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17881, 17881] 00020000:00200000:1.0:1789760785.949360:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73240576 chunk: [73240576, 73244672) 73244672, 4096 00020000:00200000:1.0:1789760785.949365:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.949386:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17881, 17881] 00000008:00010000:1.0:1789760785.949411:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.949417:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.949431:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73240576, 73244672) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.949444:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73240576=0x45d9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.949515:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.949519:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73244672 00000020:00010000:1.0:1789760785.949526:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.949529:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.949532:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.949536:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0c68 (ffff93f84418aab8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.949539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.949546:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.949553:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.949572:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760785.949717:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73240576<>73244672 00000100:00100000:0.0:1789760785.949942:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760785.949962:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565519232:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760785.958623:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985189 gen 3 00000008:00010000:0.0:1789760785.958719:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760785.958728:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760785.958739:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760785.958766:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760785.958796:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565519232:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.959186:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17881, 17881] 00000080:00200000:3.0:1789760785.959235:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.959241:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.959258:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73244672 i_blocks 143080 00000020:00200000:3.0:1789760785.959261:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.959270:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760785.959329:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73244672, count: 4096 00000080:00200000:3.0:1789760785.959338:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73244672, bytes: 4096 00000080:00200000:3.0:1789760785.959349:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760785.959358:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760785.959360:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73244672, 73248768) 0 0 00000080:00200000:3.0:1789760785.959365:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760785.959384:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17882, 17882] 00020000:00200000:3.0:1789760785.959402:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73244672 chunk: [73244672, 73248768) 73248768, 4096 00020000:00200000:3.0:1789760785.959407:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760785.959429:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17882, 17882] 00000008:00010000:3.0:1789760785.959448:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760785.959463:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760785.959476:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73244672, 73248768) append 0 append lockpos 0 00000080:00200000:3.0:1789760785.959490:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73244672=0x45da000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760785.959550:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760785.959553:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73248768 00000020:00010000:3.0:1789760785.959571:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760785.959575:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.959578:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760785.959581:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760785.959584:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760785.959592:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760785.959599:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760785.959616:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760785.959776:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 73244672<>73248768 00000100:00100000:3.0:1789760785.959823:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760785.959852:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565519360:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760785.968321:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985190 gen 3 00000008:00010000:3.0:1789760785.968376:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:3.0:1789760785.968385:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760785.968409:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760785.968435:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:3.0:1789760785.968478:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565519360:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760785.968494:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17882, 17882] 00000080:00200000:2.0:1789760785.968515:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760785.968521:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760785.968536:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73248768 i_blocks 143088 00000020:00200000:2.0:1789760785.968541:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760785.968552:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.968603:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73248768, count: 4096 00000080:00200000:2.0:1789760785.968613:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73248768, bytes: 4096 00000080:00200000:2.0:1789760785.968624:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.968633:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.968635:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73248768, 73252864) 0 0 00000080:00200000:2.0:1789760785.968640:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.968691:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17883, 17883] 00020000:00200000:2.0:1789760785.968701:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73248768 chunk: [73248768, 73252864) 73252864, 4096 00020000:00200000:2.0:1789760785.968706:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.968728:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17883, 17883] 00000008:00010000:2.0:1789760785.968750:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.968757:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.968769:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73248768, 73252864) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.968783:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73248768=0x45db000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.968839:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.968841:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73252864 00000020:00010000:2.0:1789760785.968851:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.968854:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.968858:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.968861:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.968866:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.968874:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.968881:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.968897:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760785.969050:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 73248768<>73252864 00000100:00100000:2.0:1789760785.969104:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760785.969122:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565519488:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.975992:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985191 gen 3 00000008:00010000:2.0:1789760785.976043:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:2.0:1789760785.976052:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.976064:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.976091:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000080:00200000:3.0:1789760785.976118:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17883, 17883] 00000100:00100000:2.0:1789760785.976125:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565519488:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760785.976138:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760785.976145:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760785.976160:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73252864 i_blocks 143096 00000020:00200000:3.0:1789760785.976164:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760785.976175:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760785.976483:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73252864, count: 4096 00000080:00200000:2.0:1789760785.976995:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73252864, bytes: 4096 00000080:00200000:2.0:1789760785.977075:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760785.977085:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760785.977088:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73252864, 73256960) 0 0 00000080:00200000:2.0:1789760785.977093:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760785.977113:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17884, 17884] 00020000:00200000:2.0:1789760785.977939:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73252864 chunk: [73252864, 73256960) 73256960, 4096 00020000:00200000:2.0:1789760785.977949:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760785.978021:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17884, 17884] 00000008:00010000:2.0:1789760785.978040:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760785.978046:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760785.978059:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73252864, 73256960) append 0 append lockpos 0 00000080:00200000:2.0:1789760785.978074:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73252864=0x45dc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760785.978206:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760785.978210:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73256960 00000020:00010000:2.0:1789760785.978219:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760785.978222:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.978225:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760785.978229:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760785.978232:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760785.978243:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760785.978251:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760785.978297:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760785.978432:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 73252864<>73256960 00000100:00100000:2.0:1789760785.978544:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565519616:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760785.993028:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985192 gen 3 00000008:00010000:2.0:1789760785.993081:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760785.993090:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760785.993102:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760785.993141:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760785.993172:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565519616:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760785.993767:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17884, 17884] 00000080:00200000:1.0:1789760785.993787:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760785.993794:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760785.993813:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73256960 i_blocks 143104 00000020:00200000:1.0:1789760785.993818:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760785.993830:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760785.993872:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73256960, count: 4096 00000080:00200000:1.0:1789760785.993879:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73256960, bytes: 4096 00000080:00200000:1.0:1789760785.993894:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760785.993905:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760785.993907:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73256960, 73261056) 0 0 00000080:00200000:1.0:1789760785.993912:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760785.993930:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17885, 17885] 00020000:00200000:1.0:1789760785.993939:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73256960 chunk: [73256960, 73261056) 73261056, 4096 00020000:00200000:1.0:1789760785.993943:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760785.993966:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17885, 17885] 00000008:00010000:1.0:1789760785.993985:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760785.993992:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760785.994003:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73256960, 73261056) append 0 append lockpos 0 00000080:00200000:1.0:1789760785.994017:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73256960=0x45dd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760785.994079:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760785.994082:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73261056 00000020:00010000:1.0:1789760785.994091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760785.994095:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.994098:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760785.994102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760785.994105:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760785.994112:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760785.994119:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760785.994249:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760785.994386:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73256960<>73261056 00000100:00100000:0.0:1789760785.994639:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565519744:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.005081:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985193 gen 3 00000008:00010000:1.0:1789760786.005135:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:1.0:1789760786.005144:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.005155:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.005251:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:1.0:1789760786.005284:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565519744:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.007913:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17885, 17885] 00000080:00200000:2.0:1789760786.007941:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.007948:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.007970:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73261056 i_blocks 143112 00000020:00200000:2.0:1789760786.007975:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.007994:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.008033:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73261056, count: 4096 00000080:00200000:2.0:1789760786.008042:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73261056, bytes: 4096 00000080:00200000:2.0:1789760786.008059:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.008069:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.008072:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73261056, 73265152) 0 0 00000080:00200000:2.0:1789760786.008077:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.008095:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17886, 17886] 00020000:00200000:2.0:1789760786.008106:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73261056 chunk: [73261056, 73265152) 73265152, 4096 00020000:00200000:2.0:1789760786.008111:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.008133:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17886, 17886] 00000008:00010000:2.0:1789760786.008152:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.008159:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.008171:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73261056, 73265152) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.008188:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73261056=0x45de000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.008326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.008329:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73265152 00000020:00010000:2.0:1789760786.008337:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.008341:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.008344:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.008347:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.008350:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.008358:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.008364:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.008396:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760786.008546:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 73261056<>73265152 00000100:00100000:3.0:1789760786.008711:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565519872:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.018815:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985194 gen 3 00000008:00010000:3.0:1789760786.018865:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760786.018873:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.018885:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.018974:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760786.019008:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565519872:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.019486:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17886, 17886] 00000080:00200000:2.0:1789760786.019505:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.019511:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.019528:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73265152 i_blocks 143120 00000020:00200000:2.0:1789760786.019532:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.019612:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.019862:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73265152, count: 4096 00000080:00200000:2.0:1789760786.019872:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73265152, bytes: 4096 00000080:00200000:2.0:1789760786.019884:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.019893:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.019895:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73265152, 73269248) 0 0 00000080:00200000:2.0:1789760786.019900:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.020887:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17887, 17887] 00020000:00200000:2.0:1789760786.020902:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73265152 chunk: [73265152, 73269248) 73269248, 4096 00020000:00200000:2.0:1789760786.020917:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.020943:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17887, 17887] 00000008:00010000:2.0:1789760786.020964:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.020971:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.020983:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73265152, 73269248) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.020998:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73265152=0x45df000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.022798:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.022807:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73269248 00000020:00010000:2.0:1789760786.022820:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.022825:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.022829:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.022832:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.022836:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.022849:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.022858:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.022888:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760786.023017:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 73265152<>73269248 00000100:00100000:3.0:1789760786.023117:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565520000:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.045428:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985195 gen 3 00000008:00010000:3.0:1789760786.045496:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760786.045504:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.045516:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.045538:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760786.045576:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565520000:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760786.045724:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17887, 17887] 00000080:00200000:1.0:1789760786.045744:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.045750:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.045769:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73269248 i_blocks 143128 00000020:00200000:1.0:1789760786.045773:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.045783:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.045819:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73269248, count: 4096 00000080:00200000:1.0:1789760786.045828:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73269248, bytes: 4096 00000080:00200000:1.0:1789760786.045842:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.045851:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.045854:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73269248, 73273344) 0 0 00000080:00200000:1.0:1789760786.045859:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.045877:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17888, 17888] 00020000:00200000:1.0:1789760786.045886:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73269248 chunk: [73269248, 73273344) 73273344, 4096 00020000:00200000:1.0:1789760786.045890:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.045912:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17888, 17888] 00000008:00010000:1.0:1789760786.045933:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.045940:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.045952:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73269248, 73273344) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.045969:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73269248=0x45e0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.046032:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.046035:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73273344 00000020:00010000:1.0:1789760786.046045:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.046048:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.046051:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.046054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.046057:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.046064:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.046071:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760786.048604:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:3.0:1789760786.048785:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 73269248<>73273344 00000100:00100000:3.0:1789760786.048859:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760786.048887:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565520128:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.061980:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985196 gen 3 00000008:00010000:3.0:1789760786.062032:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:3.0:1789760786.062041:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.062052:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.062130:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:3.0:1789760786.062162:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565520128:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760786.062257:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17888, 17888] 00000080:00200000:1.0:1789760786.062274:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.062281:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.062297:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73273344 i_blocks 143136 00000020:00200000:1.0:1789760786.062301:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.062312:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.062347:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73273344, count: 4096 00000080:00200000:1.0:1789760786.062355:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73273344, bytes: 4096 00000080:00200000:1.0:1789760786.062364:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.062372:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.062374:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73273344, 73277440) 0 0 00000080:00200000:1.0:1789760786.062379:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.062463:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17889, 17889] 00020000:00200000:1.0:1789760786.062473:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73273344 chunk: [73273344, 73277440) 73277440, 4096 00020000:00200000:1.0:1789760786.062478:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.062499:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17889, 17889] 00000008:00010000:1.0:1789760786.062519:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.062525:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.062538:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73273344, 73277440) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.062552:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73273344=0x45e1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.062611:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.062614:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73277440 00000020:00010000:1.0:1789760786.062623:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.062626:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.062630:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.062634:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.062637:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.062906:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.062914:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.062933:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760786.063059:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73273344<>73277440 00000100:00100000:0.0:1789760786.063130:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565520256:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.072960:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985197 gen 3 00000008:00010000:1.0:1789760786.073012:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760786.073020:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.073033:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.073118:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760786.073149:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565520256:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.073223:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17889, 17889] 00000080:00200000:2.0:1789760786.073239:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.073245:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.073263:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73277440 i_blocks 143144 00000020:00200000:2.0:1789760786.073266:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.073276:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.073308:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73277440, count: 4096 00000080:00200000:2.0:1789760786.073315:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73277440, bytes: 4096 00000080:00200000:2.0:1789760786.073324:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.073332:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.073334:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73277440, 73281536) 0 0 00000080:00200000:2.0:1789760786.073338:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.073356:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17890, 17890] 00020000:00200000:2.0:1789760786.073363:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73277440 chunk: [73277440, 73281536) 73281536, 4096 00020000:00200000:2.0:1789760786.073366:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.073383:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17890, 17890] 00000008:00010000:2.0:1789760786.073401:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.073407:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.073420:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73277440, 73281536) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.073436:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73277440=0x45e2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.073575:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.073578:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73281536 00000020:00010000:2.0:1789760786.073586:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.073590:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.073593:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.073596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.073600:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.073607:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.073615:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.073632:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760786.073805:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 73277440<>73281536 00000100:00100000:2.0:1789760786.073849:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760786.073866:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565520384:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.082724:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985198 gen 3 00000008:00010000:2.0:1789760786.082770:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760786.082778:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.082788:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.082811:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:2.0:1789760786.082840:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565520384:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760786.082957:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17890, 17890] 00000080:00200000:3.0:1789760786.083006:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760786.083012:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760786.083027:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73281536 i_blocks 143152 00000020:00200000:3.0:1789760786.083031:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760786.083040:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760786.083087:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73281536, count: 4096 00000080:00200000:3.0:1789760786.083095:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73281536, bytes: 4096 00000080:00200000:3.0:1789760786.083106:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760786.083115:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760786.083117:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73281536, 73285632) 0 0 00000080:00200000:3.0:1789760786.083122:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760786.083139:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17891, 17891] 00020000:00200000:3.0:1789760786.083149:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73281536 chunk: [73281536, 73285632) 73285632, 4096 00020000:00200000:3.0:1789760786.083154:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760786.083175:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17891, 17891] 00000008:00010000:3.0:1789760786.083192:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760786.083198:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760786.083221:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73281536, 73285632) append 0 append lockpos 0 00000080:00200000:3.0:1789760786.083236:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73281536=0x45e3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760786.083292:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760786.083295:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73285632 00000020:00010000:3.0:1789760786.083316:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760786.083320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760786.083323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760786.083326:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760786.083328:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760786.083337:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760786.083343:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760786.083359:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760786.083493:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 73281536<>73285632 00000100:00100000:3.0:1789760786.083754:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760786.083771:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565520512:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.092828:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985199 gen 3 00000008:00010000:3.0:1789760786.092890:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:3.0:1789760786.092899:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.092910:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.092939:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:3.0:1789760786.092978:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565520512:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760786.093204:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17891, 17891] 00000080:00200000:1.0:1789760786.093222:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.093227:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.093241:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73285632 i_blocks 143160 00000020:00200000:1.0:1789760786.093244:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.093255:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.093289:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73285632, count: 4096 00000080:00200000:1.0:1789760786.093296:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73285632, bytes: 4096 00000080:00200000:1.0:1789760786.093306:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.093314:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.093316:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73285632, 73289728) 0 0 00000080:00200000:1.0:1789760786.093321:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.093336:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17892, 17892] 00020000:00200000:1.0:1789760786.093344:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73285632 chunk: [73285632, 73289728) 73289728, 4096 00020000:00200000:1.0:1789760786.093348:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.093367:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17892, 17892] 00000008:00010000:1.0:1789760786.093383:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.093389:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.093399:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73285632, 73289728) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.093412:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73285632=0x45e4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.093948:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.093953:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73289728 00000020:00010000:1.0:1789760786.093963:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.094021:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.094025:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.094029:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.094032:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.094041:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.094047:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.094069:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760786.094243:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73285632<>73289728 00000100:00100000:0.0:1789760786.094313:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565520640:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.105912:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985200 gen 3 00000008:00010000:1.0:1789760786.105961:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760786.105979:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.105991:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.106010:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760786.106041:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565520640:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760786.106300:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17892, 17892] 00000080:00200000:3.0:1789760786.106317:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760786.106323:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760786.106336:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73289728 i_blocks 143168 00000020:00200000:3.0:1789760786.106340:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760786.106349:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760786.106383:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73289728, count: 4096 00000080:00200000:3.0:1789760786.106400:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73289728, bytes: 4096 00000080:00200000:3.0:1789760786.106411:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760786.106419:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760786.106422:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73289728, 73293824) 0 0 00000080:00200000:3.0:1789760786.106426:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760786.106442:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17893, 17893] 00020000:00200000:3.0:1789760786.106458:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73289728 chunk: [73289728, 73293824) 73293824, 4096 00020000:00200000:3.0:1789760786.106463:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760786.106628:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17893, 17893] 00000008:00010000:2.0:1789760786.107006:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.107014:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567ea30, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.107029:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73289728, 73293824) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.107044:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73289728=0x45e5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.107101:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.107103:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73293824 00000020:00010000:2.0:1789760786.107111:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.107115:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.107117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567ea30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.107121:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.107124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567ea30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.107132:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.107137:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.107151:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760786.107266:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 73289728<>73293824 00000100:00100000:2.0:1789760786.107311:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760786.107329:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565520768:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.119247:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985201 gen 3 00000008:00010000:2.0:1789760786.119301:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760786.119312:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.119324:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.119354:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:2.0:1789760786.119410:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565520768:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760786.119980:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17893, 17893] 00000080:00200000:1.0:1789760786.120001:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.120007:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.120024:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73293824 i_blocks 143176 00000020:00200000:1.0:1789760786.120029:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.120041:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.120080:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73293824, count: 4096 00000080:00200000:1.0:1789760786.120089:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73293824, bytes: 4096 00000080:00200000:1.0:1789760786.120102:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.120113:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.120115:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73293824, 73297920) 0 0 00000080:00200000:1.0:1789760786.120120:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.120137:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17894, 17894] 00020000:00200000:1.0:1789760786.120147:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73293824 chunk: [73293824, 73297920) 73297920, 4096 00020000:00200000:1.0:1789760786.120151:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.120174:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17894, 17894] 00000008:00010000:1.0:1789760786.120192:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.120198:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.120211:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73293824, 73297920) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.120227:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73293824=0x45e6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.120289:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.120292:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73297920 00000020:00010000:1.0:1789760786.120301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.120305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.120308:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.120312:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.120314:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.120322:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.120330:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.120349:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760786.122219:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73293824<>73297920 00000100:00100000:1.0:1789760786.122309:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565520896:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.136098:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985202 gen 3 00000008:00010000:1.0:1789760786.136150:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760786.136158:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.136169:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.136236:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760786.136266:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565520896:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.137524:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17894, 17894] 00000080:00200000:0.0:1789760786.137545:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.137552:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.137571:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73297920 i_blocks 143184 00000020:00200000:0.0:1789760786.137575:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.137591:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.137629:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73297920, count: 4096 00000080:00200000:0.0:1789760786.137638:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73297920, bytes: 4096 00000080:00200000:0.0:1789760786.137788:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.137806:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.137811:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73297920, 73302016) 0 0 00000080:00200000:0.0:1789760786.137818:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.137836:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17895, 17895] 00020000:00200000:0.0:1789760786.137845:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73297920 chunk: [73297920, 73302016) 73302016, 4096 00020000:00200000:0.0:1789760786.137850:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.137877:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17895, 17895] 00000008:00010000:0.0:1789760786.137896:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.137903:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.137916:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73297920, 73302016) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.137931:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73297920=0x45e7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.138004:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.138007:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73302016 00000020:00010000:0.0:1789760786.138016:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.138019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.138022:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.138026:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.138029:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.138038:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.138046:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.138073:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760786.138209:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43100 - object 0x240000400:5 offset 73297920<>73302016 00000100:00100000:1.0:1789760786.138346:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565521024:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.150125:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985203 gen 3 00000008:00010000:1.0:1789760786.150170:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760786.150178:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.150189:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.150210:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760786.150239:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565521024:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.150250:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17895, 17895] 00000080:00200000:0.0:1789760786.150270:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.150386:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.150411:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73302016 i_blocks 143192 00000020:00200000:0.0:1789760786.150415:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.150427:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.150474:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73302016, count: 4096 00000080:00200000:0.0:1789760786.150482:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73302016, bytes: 4096 00000080:00200000:0.0:1789760786.150500:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.150516:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.150519:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73302016, 73306112) 0 0 00000080:00200000:0.0:1789760786.150524:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.150542:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17896, 17896] 00020000:00200000:0.0:1789760786.150551:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73302016 chunk: [73302016, 73306112) 73306112, 4096 00020000:00200000:0.0:1789760786.150556:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.150577:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17896, 17896] 00000008:00010000:0.0:1789760786.150596:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.150602:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.150613:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73302016, 73306112) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.150628:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73302016=0x45e8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.150731:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.150734:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73306112 00000020:00010000:0.0:1789760786.150742:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.150746:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.150749:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.150753:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.150757:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.150765:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.150771:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.150790:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760786.150924:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41f80 - object 0x240000400:5 offset 73302016<>73306112 00000100:00100000:0.0:1789760786.150974:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565521152:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.164604:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985204 gen 3 00000008:00010000:1.0:1789760786.164710:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760786.164721:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.164732:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.164763:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760786.164797:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565521152:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.165106:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17896, 17896] 00000080:00200000:2.0:1789760786.165122:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.165128:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.165144:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73306112 i_blocks 143200 00000020:00200000:2.0:1789760786.165148:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.165160:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.165195:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73306112, count: 4096 00000080:00200000:2.0:1789760786.165203:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73306112, bytes: 4096 00000080:00200000:2.0:1789760786.165215:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.165224:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.165227:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73306112, 73310208) 0 0 00000080:00200000:2.0:1789760786.165231:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.165248:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17897, 17897] 00020000:00200000:2.0:1789760786.165257:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73306112 chunk: [73306112, 73310208) 73310208, 4096 00020000:00200000:2.0:1789760786.165262:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.165283:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17897, 17897] 00000008:00010000:2.0:1789760786.165302:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.165308:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.165320:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73306112, 73310208) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.165335:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73306112=0x45e9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.165396:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.165399:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73310208 00000020:00010000:2.0:1789760786.165408:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.165411:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.165415:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.165418:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.165421:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.165429:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.165436:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.165466:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760786.165716:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 73306112<>73310208 00000100:00100000:3.0:1789760786.166083:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565521280:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.178689:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985205 gen 3 00000008:00010000:3.0:1789760786.178730:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760786.178736:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.178744:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.178762:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760786.178784:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565521280:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.179545:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17897, 17897] 00000080:00200000:2.0:1789760786.179569:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.179576:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.179594:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73310208 i_blocks 143208 00000020:00200000:2.0:1789760786.179599:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.179614:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.179687:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73310208, count: 4096 00000080:00200000:2.0:1789760786.179696:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73310208, bytes: 4096 00000080:00200000:2.0:1789760786.179710:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.179720:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.179722:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73310208, 73314304) 0 0 00000080:00200000:2.0:1789760786.179727:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.179746:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17898, 17898] 00020000:00200000:2.0:1789760786.179756:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73310208 chunk: [73310208, 73314304) 73314304, 4096 00020000:00200000:2.0:1789760786.179761:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.179916:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17898, 17898] 00000008:00010000:2.0:1789760786.180218:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.180224:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.180247:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73310208, 73314304) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.180261:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73310208=0x45ea000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.180623:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.180627:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73314304 00000020:00010000:2.0:1789760786.180637:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.180641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.180672:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.180676:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.180679:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.180689:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.180696:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.180722:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760786.180937:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 73310208<>73314304 00000100:00100000:3.0:1789760786.181172:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565521408:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.194773:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985206 gen 3 00000008:00010000:2.0:1789760786.194893:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760786.194902:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.194913:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.195000:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760786.195030:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565521408:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760786.195463:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17898, 17898] 00000080:00200000:1.0:1789760786.195480:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.195486:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.195500:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73314304 i_blocks 143216 00000020:00200000:1.0:1789760786.195504:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.195515:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.195551:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73314304, count: 4096 00000080:00200000:1.0:1789760786.195558:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73314304, bytes: 4096 00000080:00200000:1.0:1789760786.195573:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.195581:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.195584:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73314304, 73318400) 0 0 00000080:00200000:1.0:1789760786.195589:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.195607:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17899, 17899] 00020000:00200000:1.0:1789760786.195616:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73314304 chunk: [73314304, 73318400) 73318400, 4096 00020000:00200000:1.0:1789760786.195620:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.195758:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17899, 17899] 00000008:00010000:1.0:1789760786.195783:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.195790:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.195801:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73314304, 73318400) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.195815:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73314304=0x45eb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.196027:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.196031:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73318400 00000020:00010000:1.0:1789760786.196040:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.196044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.196046:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.196050:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.196053:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.196061:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.196069:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.196090:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760786.196217:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73314304<>73318400 00000100:00100000:1.0:1789760786.196502:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565521536:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.209008:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985207 gen 3 00000008:00010000:1.0:1789760786.209058:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760786.209067:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.209079:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.209100:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760786.209130:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565521536:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.209357:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17899, 17899] 00000080:00200000:0.0:1789760786.209376:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.209381:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.209464:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73318400 i_blocks 143224 00000020:00200000:0.0:1789760786.209469:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.209480:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.209579:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73318400, count: 4096 00000080:00200000:0.0:1789760786.209589:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73318400, bytes: 4096 00000080:00200000:0.0:1789760786.209605:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.209615:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.209617:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73318400, 73322496) 0 0 00000080:00200000:0.0:1789760786.209622:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.209638:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17900, 17900] 00020000:00200000:0.0:1789760786.209812:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73318400 chunk: [73318400, 73322496) 73322496, 4096 00020000:00200000:0.0:1789760786.209818:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.209953:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17900, 17900] 00000008:00010000:0.0:1789760786.209974:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.209980:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.209993:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73318400, 73322496) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.210009:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73318400=0x45ec000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.210077:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.210081:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73322496 00000020:00010000:0.0:1789760786.210090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.210094:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.210098:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.210101:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.210104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.210112:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.210120:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.210141:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760786.210637:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 73318400<>73322496 00000100:00100000:1.0:1789760786.211665:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565521664:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.220766:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985208 gen 3 00000008:00010000:1.0:1789760786.220824:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760786.220833:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.220844:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.220866:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:1.0:1789760786.220896:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565521664:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.221190:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17900, 17900] 00000080:00200000:0.0:1789760786.221207:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.221213:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.221229:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73322496 i_blocks 143232 00000020:00200000:0.0:1789760786.221232:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.221363:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.221468:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73322496, count: 4096 00000080:00200000:0.0:1789760786.221477:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73322496, bytes: 4096 00000080:00200000:0.0:1789760786.221489:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.221497:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.221500:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73322496, 73326592) 0 0 00000080:00200000:0.0:1789760786.221505:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.221522:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17901, 17901] 00020000:00200000:0.0:1789760786.221533:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73322496 chunk: [73322496, 73326592) 73326592, 4096 00020000:00200000:0.0:1789760786.221537:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.221558:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17901, 17901] 00000008:00010000:0.0:1789760786.221577:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.221584:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.221596:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73322496, 73326592) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.221611:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73322496=0x45ed000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.221979:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.221984:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73326592 00000020:00010000:0.0:1789760786.221993:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.221997:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.222001:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.222004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.222007:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.222016:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.222024:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.222045:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d300 00000008:00100000:0.0:1789760786.222173:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 73322496<>73326592 00000100:00100000:1.0:1789760786.222320:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565521792:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.237511:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985209 gen 3 00000008:00010000:1.0:1789760786.237564:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760786.237572:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.237584:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.237785:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760786.237817:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565521792:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.239398:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17901, 17901] 00000080:00200000:0.0:1789760786.239423:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.239430:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.239458:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73326592 i_blocks 143240 00000020:00200000:0.0:1789760786.239464:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.239480:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.239514:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73326592, count: 4096 00000080:00200000:0.0:1789760786.239522:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73326592, bytes: 4096 00000080:00200000:0.0:1789760786.239537:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.239546:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.239548:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73326592, 73330688) 0 0 00000080:00200000:0.0:1789760786.239553:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.239571:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17902, 17902] 00020000:00200000:0.0:1789760786.239581:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73326592 chunk: [73326592, 73330688) 73330688, 4096 00020000:00200000:0.0:1789760786.239586:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.239609:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17902, 17902] 00000008:00010000:0.0:1789760786.239628:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.239635:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.239779:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73326592, 73330688) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.239797:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73326592=0x45ee000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.239869:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.239873:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73330688 00000020:00010000:0.0:1789760786.239883:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.239887:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.239891:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.239894:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.239897:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.239906:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.239913:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.239944:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760786.240080:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 73326592<>73330688 00000100:00100000:1.0:1789760786.240410:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565521920:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.256044:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985210 gen 3 00000008:00010000:1.0:1789760786.256098:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760786.256107:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.256118:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.256142:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:1.0:1789760786.256171:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565521920:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.256376:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17902, 17902] 00000080:00200000:0.0:1789760786.256394:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.256400:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.256416:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73330688 i_blocks 143248 00000020:00200000:0.0:1789760786.256419:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.256431:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.256475:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73330688, count: 4096 00000080:00200000:0.0:1789760786.256485:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73330688, bytes: 4096 00000080:00200000:0.0:1789760786.256498:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.256506:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.256509:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73330688, 73334784) 0 0 00000080:00200000:0.0:1789760786.256513:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.256532:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17903, 17903] 00020000:00200000:0.0:1789760786.256542:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73330688 chunk: [73330688, 73334784) 73334784, 4096 00020000:00200000:0.0:1789760786.256548:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.256570:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17903, 17903] 00000008:00010000:0.0:1789760786.256589:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.256595:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.256607:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73330688, 73334784) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.256621:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73330688=0x45ef000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.256709:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.256712:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73334784 00000020:00010000:0.0:1789760786.256720:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.256724:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.256727:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.256731:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.256734:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.256743:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.256749:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.256965:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:1.0:1789760786.257103:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73330688<>73334784 00000100:00100000:1.0:1789760786.257164:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565522048:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.271174:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985211 gen 3 00000008:00010000:1.0:1789760786.271228:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760786.271237:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.271249:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.271270:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760786.271300:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565522048:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.271412:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17903, 17903] 00000080:00200000:0.0:1789760786.271428:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.271434:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.271459:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73334784 i_blocks 143256 00000020:00200000:0.0:1789760786.271463:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.271475:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.271512:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73334784, count: 4096 00000080:00200000:0.0:1789760786.271571:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73334784, bytes: 4096 00000080:00200000:0.0:1789760786.271583:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.271593:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.271596:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73334784, 73338880) 0 0 00000080:00200000:0.0:1789760786.271601:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.271721:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17904, 17904] 00020000:00200000:0.0:1789760786.271731:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73334784 chunk: [73334784, 73338880) 73338880, 4096 00020000:00200000:0.0:1789760786.271736:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.271757:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17904, 17904] 00000008:00010000:0.0:1789760786.271778:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.271784:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.271796:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73334784, 73338880) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.271811:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73334784=0x45f0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.272028:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.272032:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73338880 00000020:00010000:0.0:1789760786.272040:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.272043:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.272046:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.272050:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.272054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.272062:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.272069:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.272089:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760786.272234:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 73334784<>73338880 00000100:00100000:1.0:1789760786.272316:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565522176:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.283759:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985212 gen 3 00000008:00010000:1.0:1789760786.283817:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760786.283834:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.283845:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.283868:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760786.283901:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565522176:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.284013:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17904, 17904] 00000080:00200000:0.0:1789760786.284033:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.284038:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.284055:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73338880 i_blocks 143264 00000020:00200000:0.0:1789760786.284059:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.284069:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.284106:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73338880, count: 4096 00000080:00200000:0.0:1789760786.284113:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73338880, bytes: 4096 00000080:00200000:0.0:1789760786.284124:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.284133:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.284136:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73338880, 73342976) 0 0 00000080:00200000:0.0:1789760786.284141:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.284159:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17905, 17905] 00020000:00200000:0.0:1789760786.284167:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73338880 chunk: [73338880, 73342976) 73342976, 4096 00020000:00200000:0.0:1789760786.284174:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.284206:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17905, 17905] 00000008:00010000:0.0:1789760786.284226:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.284232:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.284244:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73338880, 73342976) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.284258:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73338880=0x45f1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.284316:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.284319:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73342976 00000020:00010000:0.0:1789760786.284328:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.284331:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.284334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.284338:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.284343:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.284352:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.284359:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.284375:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760786.284512:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 73338880<>73342976 00000100:00100000:1.0:1789760786.284875:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565522304:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.299147:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985213 gen 3 00000008:00010000:1.0:1789760786.299202:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760786.299211:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.299223:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.299245:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:1.0:1789760786.299278:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565522304:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.299404:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17905, 17905] 00000080:00200000:0.0:1789760786.299424:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.299430:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.299446:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73342976 i_blocks 143272 00000020:00200000:0.0:1789760786.300207:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.300224:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.300266:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73342976, count: 4096 00000080:00200000:0.0:1789760786.300276:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73342976, bytes: 4096 00000080:00200000:0.0:1789760786.300290:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.300298:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.300301:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73342976, 73347072) 0 0 00000080:00200000:0.0:1789760786.300306:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.300326:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17906, 17906] 00020000:00200000:0.0:1789760786.300337:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73342976 chunk: [73342976, 73347072) 73347072, 4096 00020000:00200000:0.0:1789760786.300342:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.300363:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17906, 17906] 00000008:00010000:0.0:1789760786.300381:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.300683:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.300699:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73342976, 73347072) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.300714:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73342976=0x45f2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.300792:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.300795:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73347072 00000020:00010000:0.0:1789760786.300804:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.300808:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.300811:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.300815:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.300818:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.300937:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.300945:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.300976:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760786.301113:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 73342976<>73347072 00000100:00100000:1.0:1789760786.301352:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565522432:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.314780:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985214 gen 3 00000008:00010000:1.0:1789760786.315412:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760786.315421:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.315432:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.315501:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760786.315535:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565522432:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.315869:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17906, 17906] 00000080:00200000:0.0:1789760786.315888:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.315894:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.315912:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73347072 i_blocks 143280 00000020:00200000:0.0:1789760786.315916:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.315928:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.315962:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73347072, count: 4096 00000080:00200000:0.0:1789760786.315969:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73347072, bytes: 4096 00000080:00200000:0.0:1789760786.316078:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.316088:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.316090:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73347072, 73351168) 0 0 00000080:00200000:0.0:1789760786.316095:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.316115:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17907, 17907] 00020000:00200000:0.0:1789760786.316124:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73347072 chunk: [73347072, 73351168) 73351168, 4096 00020000:00200000:0.0:1789760786.316263:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.316285:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17907, 17907] 00000008:00010000:0.0:1789760786.316306:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.316313:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.316329:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73347072, 73351168) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.316345:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73347072=0x45f3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.316412:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.316415:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73351168 00000020:00010000:0.0:1789760786.316424:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.316427:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.316430:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.316434:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.316438:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.316447:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.316544:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.316571:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760786.316846:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 73347072<>73351168 00000100:00100000:1.0:1789760786.316935:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565522560:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.330284:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985215 gen 3 00000008:00010000:1.0:1789760786.330336:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760786.330345:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.330356:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.330377:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000080:00200000:0.0:1789760786.330536:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17907, 17907] 00000080:00200000:0.0:1789760786.330555:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.330561:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.330578:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73351168 i_blocks 143288 00000020:00200000:0.0:1789760786.330581:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.330593:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.330629:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73351168, count: 4096 00000080:00200000:0.0:1789760786.330638:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73351168, bytes: 4096 00000080:00200000:0.0:1789760786.330696:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.330706:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.330708:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73351168, 73355264) 0 0 00000080:00200000:0.0:1789760786.330713:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.330731:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17908, 17908] 00020000:00200000:0.0:1789760786.330741:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73351168 chunk: [73351168, 73355264) 73355264, 4096 00020000:00200000:0.0:1789760786.330746:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.330768:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17908, 17908] 00000008:00010000:0.0:1789760786.330786:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.330792:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.330878:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73351168, 73355264) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.330894:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73351168=0x45f4000, pages 1 (max 131072), lockless, parallel 00000100:00100000:1.0:1789760786.331231:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565522560:192.168.203.147@tcp:4:dd.0 00000020:00200000:0.0:1789760786.334516:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.334530:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73355264 00000020:00010000:0.0:1789760786.334550:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.334555:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.334560:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.334564:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.334569:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.334584:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.334595:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.338190:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:1.0:1789760786.338353:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73351168<>73355264 00000100:00100000:0.0:1789760786.339192:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565522688:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.359101:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985216 gen 3 00000008:00010000:1.0:1789760786.359156:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760786.359165:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.359176:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.359199:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760786.359230:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565522688:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.359502:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17908, 17908] 00000080:00200000:0.0:1789760786.359520:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.359526:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.359541:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73355264 i_blocks 143296 00000020:00200000:0.0:1789760786.359545:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.359566:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.359603:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73355264, count: 4096 00000080:00200000:0.0:1789760786.359611:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73355264, bytes: 4096 00000080:00200000:0.0:1789760786.359624:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.359633:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.359636:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73355264, 73359360) 0 0 00000080:00200000:0.0:1789760786.359641:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.359696:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17909, 17909] 00020000:00200000:0.0:1789760786.359705:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73355264 chunk: [73355264, 73359360) 73359360, 4096 00020000:00200000:0.0:1789760786.359710:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.359732:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17909, 17909] 00000008:00010000:0.0:1789760786.359752:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.359759:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.359771:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73355264, 73359360) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.359785:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73355264=0x45f5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.359858:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.359861:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73359360 00000020:00010000:0.0:1789760786.359870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.359874:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.359877:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.359881:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.359885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.359892:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.359899:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.359916:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:0.0:1789760786.360053:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 73355264<>73359360 00000100:00100000:1.0:1789760786.360424:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565522816:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.375328:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985217 gen 3 00000008:00010000:1.0:1789760786.375379:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760786.375750:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.375762:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.375889:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760786.375921:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565522816:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.376008:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17909, 17909] 00000080:00200000:0.0:1789760786.376024:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.376029:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.376045:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73359360 i_blocks 143304 00000020:00200000:0.0:1789760786.376048:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.376060:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.376165:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73359360, count: 4096 00000080:00200000:0.0:1789760786.376174:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73359360, bytes: 4096 00000080:00200000:0.0:1789760786.376187:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.376196:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.376199:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73359360, 73363456) 0 0 00000080:00200000:0.0:1789760786.376203:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.376320:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17910, 17910] 00020000:00200000:0.0:1789760786.376331:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73359360 chunk: [73359360, 73363456) 73363456, 4096 00020000:00200000:0.0:1789760786.376336:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.376362:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17910, 17910] 00000008:00010000:0.0:1789760786.376382:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.380206:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.380295:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73359360, 73363456) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.380314:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73359360=0x45f6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.380897:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.380902:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73363456 00000020:00010000:0.0:1789760786.380914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.380918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.380921:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418abd8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.380925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.380929:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.380942:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.380952:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.380976:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760786.381182:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41f80 - object 0x240000400:5 offset 73359360<>73363456 00000100:00100000:1.0:1789760786.381296:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565522944:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.393868:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985218 gen 3 00000008:00010000:1.0:1789760786.393922:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760786.393932:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.393943:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.394013:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:1.0:1789760786.394044:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565522944:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.394317:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17910, 17910] 00000080:00200000:0.0:1789760786.394335:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.394341:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.394729:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73363456 i_blocks 143312 00000020:00200000:1.0:1789760786.394735:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.394750:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.394788:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73363456, count: 4096 00000080:00200000:1.0:1789760786.394868:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73363456, bytes: 4096 00000080:00200000:1.0:1789760786.394883:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.394892:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.394894:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73363456, 73367552) 0 0 00000080:00200000:1.0:1789760786.394899:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.394915:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17911, 17911] 00020000:00200000:1.0:1789760786.394925:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73363456 chunk: [73363456, 73367552) 73367552, 4096 00020000:00200000:1.0:1789760786.394930:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.394949:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17911, 17911] 00000008:00010000:1.0:1789760786.394968:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.394973:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.394985:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73363456, 73367552) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.394999:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73363456=0x45f7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.395059:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.395062:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73367552 00000020:00010000:1.0:1789760786.395071:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.395074:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.395077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.395080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.395083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.395159:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.395167:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.395185:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760786.395378:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73363456<>73367552 00000100:00100000:0.0:1789760786.395540:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565523072:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.409807:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985219 gen 3 00000008:00010000:1.0:1789760786.409859:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760786.409868:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.409880:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.409966:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760786.409997:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565523072:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.410568:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17911, 17911] 00000080:00200000:2.0:1789760786.410587:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.410593:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.410609:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73367552 i_blocks 143320 00000020:00200000:2.0:1789760786.410613:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.410633:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.410700:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73367552, count: 4096 00000080:00200000:2.0:1789760786.410708:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73367552, bytes: 4096 00000080:00200000:2.0:1789760786.410733:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.410743:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.410746:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73367552, 73371648) 0 0 00000080:00200000:2.0:1789760786.410750:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.410767:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17912, 17912] 00020000:00200000:2.0:1789760786.410776:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73367552 chunk: [73367552, 73371648) 73371648, 4096 00020000:00200000:2.0:1789760786.410781:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.410802:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17912, 17912] 00000008:00010000:2.0:1789760786.410822:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.410827:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.410840:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73367552, 73371648) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.410853:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73367552=0x45f8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.410917:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.410919:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73371648 00000020:00010000:2.0:1789760786.410928:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.410932:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.410936:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.410940:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.410943:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.410950:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.410958:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.410978:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760786.411111:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 73367552<>73371648 00000100:00100000:2.0:1789760786.411169:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760786.411191:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565523200:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.428279:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985220 gen 3 00000008:00010000:2.0:1789760786.428336:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760786.428348:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.428360:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.428378:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760786.428474:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565523200:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760786.428832:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17912, 17912] 00000080:00200000:3.0:1789760786.428850:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760786.428857:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760786.428872:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73371648 i_blocks 143328 00000020:00200000:3.0:1789760786.428876:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760786.428887:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760786.429025:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73371648, count: 4096 00000080:00200000:3.0:1789760786.429034:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73371648, bytes: 4096 00000080:00200000:3.0:1789760786.429045:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760786.429054:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760786.429056:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73371648, 73375744) 0 0 00000080:00200000:3.0:1789760786.429061:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760786.429077:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17913, 17913] 00020000:00200000:3.0:1789760786.429087:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73371648 chunk: [73371648, 73375744) 73375744, 4096 00020000:00200000:3.0:1789760786.429092:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760786.429112:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17913, 17913] 00000008:00010000:3.0:1789760786.429129:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760786.429134:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760786.429147:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73371648, 73375744) append 0 append lockpos 0 00000080:00200000:3.0:1789760786.429161:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73371648=0x45f9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760786.429219:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760786.429222:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73375744 00000020:00010000:3.0:1789760786.429230:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760786.429234:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760786.429237:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760786.429241:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760786.429243:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760786.429251:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760786.429258:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760786.429276:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760786.429466:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 73371648<>73375744 00000100:00100000:3.0:1789760786.429513:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760786.429533:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565523328:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.446533:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985221 gen 3 00000008:00010000:2.0:1789760786.446586:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760786.446595:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.446607:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.446636:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760786.446699:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565523328:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760786.446894:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17913, 17913] 00000080:00200000:1.0:1789760786.446921:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.446928:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.446944:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73375744 i_blocks 143336 00000020:00200000:1.0:1789760786.446948:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.446959:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.446994:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73375744, count: 4096 00000080:00200000:1.0:1789760786.447001:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73375744, bytes: 4096 00000080:00200000:1.0:1789760786.447011:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.447019:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.447022:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73375744, 73379840) 0 0 00000080:00200000:1.0:1789760786.447026:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.447042:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17914, 17914] 00020000:00200000:1.0:1789760786.447050:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73375744 chunk: [73375744, 73379840) 73379840, 4096 00020000:00200000:1.0:1789760786.447054:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.447074:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17914, 17914] 00000008:00010000:1.0:1789760786.447092:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.447099:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.447110:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73375744, 73379840) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.447124:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73375744=0x45fa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.447184:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.447186:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73379840 00000020:00010000:1.0:1789760786.447195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.447198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.447202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.447206:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.447209:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.447217:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.447236:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.447381:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760786.447514:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73375744<>73379840 00000100:00100000:1.0:1789760786.447567:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760786.447586:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565523456:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.470808:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985222 gen 3 00000008:00010000:1.0:1789760786.470914:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760786.470923:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.470935:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.470992:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760786.471150:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565523456:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.472110:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17914, 17914] 00000080:00200000:0.0:1789760786.472131:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.472137:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.472156:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73379840 i_blocks 143344 00000020:00200000:0.0:1789760786.472160:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.472171:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.472372:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73379840, count: 4096 00000080:00200000:0.0:1789760786.472381:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73379840, bytes: 4096 00000080:00200000:0.0:1789760786.472890:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.472904:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.472907:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73379840, 73383936) 0 0 00000080:00200000:0.0:1789760786.472913:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.472935:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17915, 17915] 00020000:00200000:0.0:1789760786.472947:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73379840 chunk: [73379840, 73383936) 73383936, 4096 00020000:00200000:0.0:1789760786.472952:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.472975:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17915, 17915] 00000008:00010000:0.0:1789760786.472994:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.473001:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.473104:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73379840, 73383936) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.473121:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73379840=0x45fb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.473204:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.473208:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73383936 00000020:00010000:0.0:1789760786.473216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.473220:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.473223:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.473227:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.473230:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.473241:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.473903:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.473937:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760786.474527:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43100 - object 0x240000400:5 offset 73379840<>73383936 00000100:00100000:1.0:1789760786.475313:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565523584:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.489788:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985223 gen 3 00000008:00010000:1.0:1789760786.490004:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760786.490013:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.490024:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.490052:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760786.490084:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565523584:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.490573:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17915, 17915] 00000080:00200000:2.0:1789760786.490590:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.490597:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.490613:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73383936 i_blocks 143352 00000020:00200000:2.0:1789760786.490616:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.490628:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.490743:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73383936, count: 4096 00000080:00200000:2.0:1789760786.490752:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73383936, bytes: 4096 00000080:00200000:2.0:1789760786.490764:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.490773:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.490776:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73383936, 73388032) 0 0 00000080:00200000:2.0:1789760786.490780:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.490796:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17916, 17916] 00020000:00200000:2.0:1789760786.490815:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73383936 chunk: [73383936, 73388032) 73388032, 4096 00020000:00200000:2.0:1789760786.490820:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.490958:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17916, 17916] 00000008:00010000:2.0:1789760786.490977:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.490982:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.490997:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73383936, 73388032) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.491011:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73383936=0x45fc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.491073:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.491076:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73388032 00000020:00010000:2.0:1789760786.491085:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.491089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.491092:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.491095:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.491099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.491108:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.491115:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.491134:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760786.491557:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 73383936<>73388032 00000100:00100000:3.0:1789760786.491661:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565523712:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.508168:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985224 gen 3 00000008:00010000:2.0:1789760786.508546:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760786.508555:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.508566:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.508665:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760786.508695:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565523712:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760786.508909:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17916, 17916] 00000080:00200000:1.0:1789760786.508929:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.508934:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.508951:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73388032 i_blocks 143360 00000020:00200000:1.0:1789760786.508955:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.508967:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.509003:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73388032, count: 4096 00000080:00200000:1.0:1789760786.509011:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73388032, bytes: 4096 00000080:00200000:1.0:1789760786.509027:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.509036:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.509038:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73388032, 73392128) 0 0 00000080:00200000:1.0:1789760786.509043:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.509060:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17917, 17917] 00020000:00200000:1.0:1789760786.509068:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73388032 chunk: [73388032, 73392128) 73392128, 4096 00020000:00200000:1.0:1789760786.509073:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.509094:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17917, 17917] 00000008:00010000:1.0:1789760786.509112:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.509118:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.509129:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73388032, 73392128) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.509143:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73388032=0x45fd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.509552:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.509556:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73392128 00000020:00010000:1.0:1789760786.509565:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.509569:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.509573:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.509576:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.509579:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.509586:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.509594:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.509615:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760786.510010:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73388032<>73392128 00000100:00100000:0.0:1789760786.510086:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565523840:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.538015:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985225 gen 3 00000008:00010000:1.0:1789760786.538069:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760786.538077:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.538089:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.538111:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760786.538146:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565523840:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.538268:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17917, 17917] 00000080:00200000:0.0:1789760786.538296:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.538303:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.538319:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73392128 i_blocks 143368 00000020:00200000:0.0:1789760786.538323:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.538335:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.538370:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73392128, count: 4096 00000080:00200000:0.0:1789760786.538377:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73392128, bytes: 4096 00000080:00200000:0.0:1789760786.538390:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.538399:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.538401:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73392128, 73396224) 0 0 00000080:00200000:0.0:1789760786.538406:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.538423:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17918, 17918] 00020000:00200000:0.0:1789760786.538432:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73392128 chunk: [73392128, 73396224) 73396224, 4096 00020000:00200000:0.0:1789760786.538437:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.538469:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17918, 17918] 00000008:00010000:0.0:1789760786.538487:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.538493:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.538505:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73392128, 73396224) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.538519:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73392128=0x45fe000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.538576:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.538579:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73396224 00000020:00010000:0.0:1789760786.538588:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.538592:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.538595:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.538599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.538602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.538611:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.538618:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.540016:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760786.540169:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40700 - object 0x240000400:5 offset 73392128<>73396224 00000100:00100000:0.0:1789760786.540231:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760786.540256:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565523968:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.558701:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985226 gen 3 00000008:00010000:1.0:1789760786.558754:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760786.558764:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.558775:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.558811:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:1.0:1789760786.558853:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565523968:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.558983:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17918, 17918] 00000080:00200000:2.0:1789760786.559000:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.559007:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.559022:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73396224 i_blocks 143376 00000020:00200000:2.0:1789760786.559026:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.559037:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.559074:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73396224, count: 4096 00000080:00200000:2.0:1789760786.559081:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73396224, bytes: 4096 00000080:00200000:2.0:1789760786.559104:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.559113:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.559115:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73396224, 73400320) 0 0 00000080:00200000:2.0:1789760786.559119:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.559138:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17919, 17919] 00020000:00200000:2.0:1789760786.559147:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73396224 chunk: [73396224, 73400320) 73400320, 4096 00020000:00200000:2.0:1789760786.559152:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.559173:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17919, 17919] 00000008:00010000:2.0:1789760786.559191:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.559197:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.559209:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73396224, 73400320) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.559224:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73396224=0x45ff000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.559297:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.559300:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73400320 00000020:00010000:2.0:1789760786.559309:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.559313:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.559316:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.559320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.559323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.559331:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.559337:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.559354:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760786.559502:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 73396224<>73400320 00000100:00100000:2.0:1789760786.559563:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760786.559581:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565524096:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.571054:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985227 gen 3 00000008:00010000:2.0:1789760786.571187:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760786.571196:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.571207:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.571350:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760786.571383:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565524096:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.574632:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17919, 17919] 00000080:00200000:0.0:1789760786.574673:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.574680:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.574699:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73400320 i_blocks 143384 00000020:00200000:0.0:1789760786.574704:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.574723:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.574758:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73400320, count: 4096 00000080:00200000:0.0:1789760786.574767:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73400320, bytes: 4096 00000080:00200000:0.0:1789760786.574782:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.574792:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.574794:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73400320, 73404416) 0 0 00000080:00200000:0.0:1789760786.574799:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.574815:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17920, 17920] 00020000:00200000:0.0:1789760786.574825:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73400320 chunk: [73400320, 73404416) 73404416, 4096 00020000:00200000:0.0:1789760786.574830:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.574852:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17920, 17920] 00000008:00010000:0.0:1789760786.574871:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.574880:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.574892:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73400320, 73404416) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.574906:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73400320=0x4600000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.574982:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.574986:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73404416 00000020:00010000:0.0:1789760786.574994:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.574998:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.575001:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.575005:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.576885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.576917:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.576931:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.576968:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760786.578603:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 73400320<>73404416 00000100:00100000:1.0:1789760786.578695:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565524224:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.600359:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985228 gen 3 00000008:00010000:1.0:1789760786.600970:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760786.600981:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.600994:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.601095:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760786.601122:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565524224:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.605143:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17920, 17920] 00000080:00200000:2.0:1789760786.605173:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.605180:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.605200:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73404416 i_blocks 143392 00000020:00200000:2.0:1789760786.605206:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.605228:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.605266:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73404416, count: 4096 00000080:00200000:2.0:1789760786.605275:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73404416, bytes: 4096 00000080:00200000:2.0:1789760786.605291:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.605303:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.605305:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73404416, 73408512) 0 0 00000080:00200000:2.0:1789760786.605310:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.605329:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17921, 17921] 00020000:00200000:2.0:1789760786.605340:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73404416 chunk: [73404416, 73408512) 73408512, 4096 00020000:00200000:2.0:1789760786.605345:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.605369:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17921, 17921] 00000008:00010000:2.0:1789760786.605390:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.605396:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.605409:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73404416, 73408512) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.605503:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73404416=0x4601000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.605789:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.605861:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73408512 00000020:00010000:2.0:1789760786.605874:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.605879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.605882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.605887:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.605890:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.605900:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.605908:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.605941:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760786.606082:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 73404416<>73408512 00000100:00100000:3.0:1789760786.606922:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565524352:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.622060:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985229 gen 3 00000008:00010000:2.0:1789760786.622183:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760786.622191:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.622202:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.622480:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760786.622522:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565524352:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760786.623050:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17921, 17921] 00000080:00200000:1.0:1789760786.623071:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.623077:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.623341:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73408512 i_blocks 143400 00000020:00200000:1.0:1789760786.623347:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.623361:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.623397:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73408512, count: 4096 00000080:00200000:1.0:1789760786.623405:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73408512, bytes: 4096 00000080:00200000:1.0:1789760786.623758:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.623769:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.623771:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73408512, 73412608) 0 0 00000080:00200000:1.0:1789760786.623777:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.623867:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17922, 17922] 00020000:00200000:1.0:1789760786.623879:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73408512 chunk: [73408512, 73412608) 73412608, 4096 00020000:00200000:1.0:1789760786.623885:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.623909:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17922, 17922] 00000008:00010000:1.0:1789760786.623928:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.623934:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.623946:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73408512, 73412608) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.623960:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73408512=0x4602000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.624035:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.624038:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73412608 00000020:00010000:1.0:1789760786.624047:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.624181:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.624187:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.624190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.624193:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.624203:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.624272:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.624303:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760786.624934:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73408512<>73412608 00000100:00100000:1.0:1789760786.624987:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760786.625011:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565524480:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.660005:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985230 gen 3 00000008:00010000:1.0:1789760786.660061:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760786.660069:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.660081:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.660245:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760786.660286:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565524480:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.660530:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17922, 17922] 00000080:00200000:0.0:1789760786.661030:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.661039:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.661157:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73412608 i_blocks 143408 00000020:00200000:0.0:1789760786.661162:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.661175:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.661208:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73412608, count: 4096 00000080:00200000:0.0:1789760786.661216:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73412608, bytes: 4096 00000080:00200000:0.0:1789760786.661231:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.661241:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.661243:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73412608, 73416704) 0 0 00000080:00200000:0.0:1789760786.661248:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.661264:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17923, 17923] 00020000:00200000:0.0:1789760786.661275:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73412608 chunk: [73412608, 73416704) 73416704, 4096 00020000:00200000:0.0:1789760786.661280:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.661301:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17923, 17923] 00000008:00010000:0.0:1789760786.661320:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.661327:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.661339:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73412608, 73416704) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.661353:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73412608=0x4603000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.661974:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.661979:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73416704 00000020:00010000:0.0:1789760786.661991:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.661994:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.661998:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.662001:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.662004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.662015:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.662023:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.662107:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760786.662242:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42680 - object 0x240000400:5 offset 73412608<>73416704 00000100:00100000:0.0:1789760786.662292:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760786.662317:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565524608:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.681398:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985231 gen 3 00000008:00010000:1.0:1789760786.681462:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760786.681470:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.681482:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.681510:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760786.681544:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565524608:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.681784:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17923, 17923] 00000080:00200000:2.0:1789760786.681800:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.681807:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.681821:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73416704 i_blocks 143416 00000020:00200000:2.0:1789760786.681825:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.681837:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.681874:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73416704, count: 4096 00000080:00200000:2.0:1789760786.681881:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73416704, bytes: 4096 00000080:00200000:2.0:1789760786.681896:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.681905:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.681907:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73416704, 73420800) 0 0 00000080:00200000:2.0:1789760786.681912:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.681928:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17924, 17924] 00020000:00200000:2.0:1789760786.681936:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73416704 chunk: [73416704, 73420800) 73420800, 4096 00020000:00200000:2.0:1789760786.681940:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.681960:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17924, 17924] 00000008:00010000:2.0:1789760786.681979:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.681985:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.681997:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73416704, 73420800) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.682011:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73416704=0x4604000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.682068:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.682071:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73420800 00000020:00010000:2.0:1789760786.682079:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.682083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.682086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.682089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.682092:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.682100:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.682106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.682123:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760786.682246:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 73416704<>73420800 00000100:00100000:2.0:1789760786.682301:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760786.682320:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565524736:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.698975:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985232 gen 3 00000008:00010000:2.0:1789760786.699025:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760786.699034:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.699045:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.699072:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760786.699101:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565524736:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.699460:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17924, 17924] 00000080:00200000:0.0:1789760786.699564:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.699571:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.699588:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73420800 i_blocks 143424 00000020:00200000:0.0:1789760786.699592:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.699602:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.699637:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73420800, count: 4096 00000080:00200000:0.0:1789760786.699671:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73420800, bytes: 4096 00000080:00200000:0.0:1789760786.699684:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.699692:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.699695:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73420800, 73424896) 0 0 00000080:00200000:0.0:1789760786.699699:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.699715:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17925, 17925] 00020000:00200000:0.0:1789760786.699724:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73420800 chunk: [73420800, 73424896) 73424896, 4096 00020000:00200000:0.0:1789760786.699730:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.699752:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17925, 17925] 00000008:00010000:0.0:1789760786.699769:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.699775:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.699787:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73420800, 73424896) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.699801:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73420800=0x4605000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.699861:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.699864:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73424896 00000020:00010000:0.0:1789760786.699873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.699877:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.699880:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.699884:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.699886:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.699895:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.699902:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.699919:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760786.700042:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 73420800<>73424896 00000100:00100000:1.0:1789760786.700127:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565524864:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.717997:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985233 gen 3 00000008:00010000:1.0:1789760786.718049:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760786.718058:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.718070:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.718094:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:1.0:1789760786.718127:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565524864:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.718371:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17925, 17925] 00000080:00200000:0.0:1789760786.718388:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.718395:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.718629:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73424896 i_blocks 143432 00000020:00200000:0.0:1789760786.718635:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.719360:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.719401:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73424896, count: 4096 00000080:00200000:0.0:1789760786.719412:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73424896, bytes: 4096 00000080:00200000:0.0:1789760786.719428:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.719438:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.719440:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73424896, 73428992) 0 0 00000080:00200000:0.0:1789760786.719456:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.719476:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17926, 17926] 00020000:00200000:0.0:1789760786.719486:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73424896 chunk: [73424896, 73428992) 73428992, 4096 00020000:00200000:0.0:1789760786.719493:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.720325:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17926, 17926] 00000008:00010000:0.0:1789760786.720354:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.720363:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.720377:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73424896, 73428992) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.720393:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73424896=0x4606000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.720477:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.720481:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73428992 00000020:00010000:0.0:1789760786.720490:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.720494:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.720497:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.720501:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.720504:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.722323:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.722346:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.722375:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d300 00000008:00100000:0.0:1789760786.723909:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 73424896<>73428992 00000100:00100000:1.0:1789760786.724193:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760786.724235:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565524992:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760786.733854:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985234 gen 3 00000008:00010000:1.0:1789760786.733912:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760786.734094:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760786.734110:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760786.734134:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760786.734167:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565524992:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.734419:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17926, 17926] 00000080:00200000:0.0:1789760786.734439:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.734444:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.734471:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73428992 i_blocks 143440 00000020:00200000:0.0:1789760786.734474:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.734486:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.734520:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73428992, count: 4096 00000080:00200000:0.0:1789760786.734527:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73428992, bytes: 4096 00000080:00200000:0.0:1789760786.734539:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.734547:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.734549:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73428992, 73433088) 0 0 00000080:00200000:0.0:1789760786.734553:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.734570:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17927, 17927] 00020000:00200000:0.0:1789760786.734578:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73428992 chunk: [73428992, 73433088) 73433088, 4096 00020000:00200000:0.0:1789760786.734584:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.734605:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17927, 17927] 00000008:00010000:0.0:1789760786.734625:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.734631:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7db30, osc lock ffff93f850af1560, flags 0x0 00000080:00200000:0.0:1789760786.734775:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73428992, 73433088) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.735173:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73428992=0x4607000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.735384:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.735387:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73433088 00000020:00010000:2.0:1789760786.735398:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.735402:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.735406:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7db30 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.735410:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.735412:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7db30 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.735423:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.735430:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.735457:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760786.735713:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 73428992<>73433088 00000100:00100000:3.0:1789760786.735889:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565525120:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.753704:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985235 gen 3 00000008:00010000:3.0:1789760786.753741:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760786.753747:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.753768:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.753792:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760786.753814:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565525120:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.753984:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17927, 17927] 00000080:00200000:2.0:1789760786.753996:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.754002:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.754861:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73433088 i_blocks 143448 00000020:00200000:2.0:1789760786.754867:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.754879:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.754907:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73433088, count: 4096 00000080:00200000:2.0:1789760786.754915:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73433088, bytes: 4096 00000080:00200000:2.0:1789760786.754926:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.754933:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.754935:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73433088, 73437184) 0 0 00000080:00200000:2.0:1789760786.754938:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.754953:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17928, 17928] 00020000:00200000:2.0:1789760786.755623:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73433088 chunk: [73433088, 73437184) 73437184, 4096 00020000:00200000:2.0:1789760786.755629:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.755675:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17928, 17928] 00000008:00010000:2.0:1789760786.755689:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.755694:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.755704:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73433088, 73437184) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.755715:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73433088=0x4608000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.755780:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.755783:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73437184 00000020:00010000:2.0:1789760786.755789:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.755793:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.755796:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.755800:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.755803:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.755810:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.755815:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.755837:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760786.755937:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 73433088<>73437184 00000100:00100000:3.0:1789760786.756433:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760786.756466:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565525248:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.771185:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985236 gen 3 00000008:00010000:3.0:1789760786.771239:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760786.771248:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.771258:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.771921:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760786.771957:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565525248:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.773882:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17928, 17928] 00000080:00200000:2.0:1789760786.773908:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.773915:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.774275:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73437184 i_blocks 143456 00000020:00200000:2.0:1789760786.774282:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.774299:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.774338:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73437184, count: 4096 00000080:00200000:2.0:1789760786.774347:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73437184, bytes: 4096 00000080:00200000:2.0:1789760786.774363:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.774372:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.774375:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73437184, 73441280) 0 0 00000080:00200000:2.0:1789760786.774379:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.774567:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17929, 17929] 00020000:00200000:2.0:1789760786.774578:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73437184 chunk: [73437184, 73441280) 73441280, 4096 00020000:00200000:2.0:1789760786.774583:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.774608:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17929, 17929] 00000008:00010000:2.0:1789760786.774858:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.774865:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.774880:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73437184, 73441280) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.774897:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73437184=0x4609000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.774968:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.774971:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73441280 00000020:00010000:2.0:1789760786.774980:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.774983:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.774987:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.774990:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.774992:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.775000:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.775007:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.775027:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760786.775163:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 73437184<>73441280 00000100:00100000:3.0:1789760786.775322:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565525376:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.793309:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985237 gen 3 00000008:00010000:3.0:1789760786.793365:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760786.793375:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.793387:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.795377:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760786.795441:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565525376:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.796765:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17929, 17929] 00000080:00200000:2.0:1789760786.796788:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.797096:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.797116:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73441280 i_blocks 143464 00000020:00200000:2.0:1789760786.797122:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.797137:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.797173:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73441280, count: 4096 00000080:00200000:2.0:1789760786.797182:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73441280, bytes: 4096 00000080:00200000:2.0:1789760786.797199:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.797210:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.797213:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73441280, 73445376) 0 0 00000080:00200000:2.0:1789760786.797217:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.797237:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17930, 17930] 00020000:00200000:2.0:1789760786.797246:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73441280 chunk: [73441280, 73445376) 73445376, 4096 00020000:00200000:2.0:1789760786.797253:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.797275:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17930, 17930] 00000008:00010000:2.0:1789760786.797294:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.797301:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.797313:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73441280, 73445376) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.797327:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73441280=0x460a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.797410:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.797413:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73445376 00000020:00010000:2.0:1789760786.797422:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.797426:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.797430:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.797433:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.797436:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.797457:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.797465:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.797493:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760786.797629:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 73441280<>73445376 00000100:00100000:3.0:1789760786.798135:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565525504:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.810772:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985238 gen 3 00000008:00010000:3.0:1789760786.810823:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760786.810831:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.810842:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.810867:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760786.810896:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565525504:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.811314:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17930, 17930] 00000080:00200000:2.0:1789760786.811331:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.811337:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.811353:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73445376 i_blocks 143472 00000020:00200000:2.0:1789760786.811356:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.811368:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.811404:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73445376, count: 4096 00000080:00200000:2.0:1789760786.811412:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73445376, bytes: 4096 00000080:00200000:2.0:1789760786.811422:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.811430:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.811433:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73445376, 73449472) 0 0 00000080:00200000:2.0:1789760786.811438:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.811502:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17931, 17931] 00020000:00200000:2.0:1789760786.811511:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73445376 chunk: [73445376, 73449472) 73449472, 4096 00020000:00200000:2.0:1789760786.811516:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.811538:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17931, 17931] 00000008:00010000:2.0:1789760786.811556:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.811562:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.811574:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73445376, 73449472) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.811588:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73445376=0x460b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.811703:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.811706:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73449472 00000020:00010000:2.0:1789760786.811725:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.811729:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.811734:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.811737:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.811740:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.811748:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.811755:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.811772:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760786.811898:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 73445376<>73449472 00000100:00100000:3.0:1789760786.812379:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565525632:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.825836:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985239 gen 3 00000008:00010000:3.0:1789760786.825892:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760786.825900:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.825914:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.825947:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760786.825982:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565525632:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.826120:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17931, 17931] 00000080:00200000:2.0:1789760786.826136:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.826142:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.826158:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73449472 i_blocks 143480 00000020:00200000:2.0:1789760786.826163:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.826172:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.826207:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73449472, count: 4096 00000080:00200000:2.0:1789760786.826214:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73449472, bytes: 4096 00000080:00200000:2.0:1789760786.826228:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.826237:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.826240:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73449472, 73453568) 0 0 00000080:00200000:2.0:1789760786.826245:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.826263:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17932, 17932] 00020000:00200000:2.0:1789760786.826273:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73449472 chunk: [73449472, 73453568) 73453568, 4096 00020000:00200000:2.0:1789760786.826277:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.826297:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17932, 17932] 00000008:00010000:2.0:1789760786.826313:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.826318:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.826329:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73449472, 73453568) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.826342:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73449472=0x460c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.826396:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.826399:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73453568 00000020:00010000:2.0:1789760786.826406:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.826410:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.826427:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.826431:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.826434:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.826441:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.826456:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.826471:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760786.826584:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 73449472<>73453568 00000100:00100000:3.0:1789760786.826771:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565525760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.843542:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985240 gen 3 00000008:00010000:3.0:1789760786.843746:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760786.843755:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760786.843767:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760786.843937:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760786.843968:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565525760:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.844066:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17932, 17932] 00000080:00200000:2.0:1789760786.844086:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.844092:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.844108:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73453568 i_blocks 143488 00000020:00200000:2.0:1789760786.844112:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.844122:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.844156:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73453568, count: 4096 00000080:00200000:2.0:1789760786.844164:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73453568, bytes: 4096 00000080:00200000:2.0:1789760786.844175:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.844184:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.844186:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73453568, 73457664) 0 0 00000080:00200000:2.0:1789760786.844190:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.844208:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17933, 17933] 00020000:00200000:2.0:1789760786.844218:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73453568 chunk: [73453568, 73457664) 73457664, 4096 00020000:00200000:2.0:1789760786.844222:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.844242:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17933, 17933] 00000008:00010000:2.0:1789760786.844261:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.844267:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.844278:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73453568, 73457664) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.844291:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73453568=0x460d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.844344:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.844347:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73457664 00000020:00010000:2.0:1789760786.844355:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.844359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.844363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.844367:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.844369:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.844375:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.844381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.844469:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760786.844666:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 73453568<>73457664 00000100:00100000:3.0:1789760786.844730:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565525888:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.863024:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985241 gen 3 00000008:00010000:2.0:1789760786.863077:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760786.863086:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.863098:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.863124:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000080:00200000:0.0:1789760786.863233:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17933, 17933] 00000080:00200000:0.0:1789760786.863252:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.863259:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.863276:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73457664 i_blocks 143496 00000020:00200000:0.0:1789760786.863280:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.863291:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.863326:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73457664, count: 4096 00000080:00200000:0.0:1789760786.863335:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73457664, bytes: 4096 00000080:00200000:0.0:1789760786.863347:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.863356:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.863358:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73457664, 73461760) 0 0 00000080:00200000:0.0:1789760786.863363:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.863389:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17934, 17934] 00020000:00200000:0.0:1789760786.863400:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73457664 chunk: [73457664, 73461760) 73461760, 4096 00020000:00200000:0.0:1789760786.863404:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.863425:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17934, 17934] 00000008:00010000:0.0:1789760786.863454:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.863460:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760786.863472:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73457664, 73461760) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.863486:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73457664=0x460e000, pages 1 (max 131072), lockless, parallel 00000100:00100000:2.0:1789760786.863737:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565525888:192.168.203.147@tcp:4:dd.0 00000020:00200000:0.0:1789760786.863862:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.863866:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73461760 00000020:00010000:0.0:1789760786.863875:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.863879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.863882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.863886:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.863889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.863898:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.863906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.863922:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760786.864046:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73457664<>73461760 00000100:00100000:1.0:1789760786.864303:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760786.864321:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565526016:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760786.876056:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985242 gen 3 00000008:00010000:0.0:1789760786.876107:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760786.876115:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760786.876137:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760786.876163:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760786.876193:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565526016:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.876425:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17934, 17934] 00000080:00200000:2.0:1789760786.876443:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.876460:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.876476:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73461760 i_blocks 143504 00000020:00200000:2.0:1789760786.876480:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.876492:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.876538:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73461760, count: 4096 00000080:00200000:2.0:1789760786.876546:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73461760, bytes: 4096 00000080:00200000:2.0:1789760786.876559:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.876568:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.876571:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73461760, 73465856) 0 0 00000080:00200000:2.0:1789760786.876575:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.876593:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17935, 17935] 00020000:00200000:2.0:1789760786.876603:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73461760 chunk: [73461760, 73465856) 73465856, 4096 00020000:00200000:2.0:1789760786.876608:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.876772:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17935, 17935] 00000008:00010000:2.0:1789760786.876792:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.876799:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.876811:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73461760, 73465856) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.876824:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73461760=0x460f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.876884:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.876888:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73465856 00000020:00010000:2.0:1789760786.876897:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.876901:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.876904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.876908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.876911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.876919:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.876925:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.876943:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760786.877072:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 73461760<>73465856 00000100:00100000:2.0:1789760786.877137:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760786.877158:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565526144:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.897699:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985243 gen 3 00000008:00010000:2.0:1789760786.897751:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760786.897760:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.897771:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.897787:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760786.897815:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565526144:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.898083:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17935, 17935] 00000080:00200000:0.0:1789760786.898100:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.898107:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.898123:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73465856 i_blocks 143512 00000020:00200000:0.0:1789760786.898127:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.898138:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.898364:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73465856, count: 4096 00000080:00200000:0.0:1789760786.898373:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73465856, bytes: 4096 00000080:00200000:0.0:1789760786.898385:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.898393:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.898396:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73465856, 73469952) 0 0 00000080:00200000:0.0:1789760786.898401:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.898417:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17936, 17936] 00020000:00200000:0.0:1789760786.898426:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73465856 chunk: [73465856, 73469952) 73469952, 4096 00020000:00200000:0.0:1789760786.898431:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.898462:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17936, 17936] 00000008:00010000:0.0:1789760786.898480:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.898485:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760786.898497:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73465856, 73469952) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.899136:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73465856=0x4610000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.899733:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.899740:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73469952 00000020:00010000:0.0:1789760786.899752:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.899757:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.899761:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.899765:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.899769:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.899777:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.899786:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.899803:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760786.900998:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73465856<>73469952 00000100:00100000:1.0:1789760786.901174:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565526272:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760786.906908:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565526528:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1789760786.907055:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789760786.907062:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:af634c28-3647-4f73-bb24-40d7878858fb:2360:1876699565526400:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1789760786.907093:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:6a301392-4ca2-4c43-8653-61d4c47997a9:2360:1876699565526656:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1789760786.912474:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565526528:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1789760786.912514:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:af634c28-3647-4f73-bb24-40d7878858fb:2360:1876699565526400:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1789760786.912528:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:6a301392-4ca2-4c43-8653-61d4c47997a9:2360:1876699565526656:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1789760786.919038:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985244 gen 3 00000008:00010000:0.0:1789760786.919088:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760786.919097:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760786.919108:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760786.919134:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760786.919164:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565526272:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760786.919279:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17936, 17936] 00000080:00200000:2.0:1789760786.919296:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760786.919302:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760786.919316:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73469952 i_blocks 143520 00000020:00200000:2.0:1789760786.919320:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760786.919331:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760786.919367:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73469952, count: 4096 00000080:00200000:2.0:1789760786.919374:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73469952, bytes: 4096 00000080:00200000:2.0:1789760786.919385:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760786.919394:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760786.919396:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73469952, 73474048) 0 0 00000080:00200000:2.0:1789760786.919400:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760786.919416:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17937, 17937] 00020000:00200000:2.0:1789760786.919426:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73469952 chunk: [73469952, 73474048) 73474048, 4096 00020000:00200000:2.0:1789760786.919430:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760786.919461:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17937, 17937] 00000008:00010000:2.0:1789760786.919480:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760786.919486:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760786.919497:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73469952, 73474048) append 0 append lockpos 0 00000080:00200000:2.0:1789760786.919510:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73469952=0x4611000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760786.919574:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760786.919578:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73474048 00000020:00010000:2.0:1789760786.919586:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760786.919590:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.919593:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760786.919596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760786.919599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760786.919607:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760786.919613:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760786.919629:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760786.919774:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 73469952<>73474048 00000100:00100000:3.0:1789760786.919858:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565526784:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760786.930467:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985245 gen 3 00000008:00010000:2.0:1789760786.930525:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760786.930534:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760786.930545:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760786.930812:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760786.930862:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565526784:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760786.931190:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17937, 17937] 00000080:00200000:0.0:1789760786.931207:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760786.931214:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760786.931231:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73474048 i_blocks 143528 00000020:00200000:0.0:1789760786.931235:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760786.931245:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760786.931283:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73474048, count: 4096 00000080:00200000:0.0:1789760786.931290:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73474048, bytes: 4096 00000080:00200000:0.0:1789760786.931302:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760786.931312:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760786.931314:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73474048, 73478144) 0 0 00000080:00200000:0.0:1789760786.931319:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760786.931337:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17938, 17938] 00020000:00200000:0.0:1789760786.931347:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73474048 chunk: [73474048, 73478144) 73478144, 4096 00020000:00200000:0.0:1789760786.931354:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760786.931376:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17938, 17938] 00000008:00010000:0.0:1789760786.931394:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760786.931400:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760786.931412:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73474048, 73478144) append 0 append lockpos 0 00000080:00200000:0.0:1789760786.931425:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73474048=0x4612000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760786.931510:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760786.931515:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73478144 00000020:00010000:0.0:1789760786.931523:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760786.931527:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.931531:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760786.931535:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760786.931539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760786.931547:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760786.931564:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760786.931584:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760786.931756:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73474048<>73478144 00000100:00100000:1.0:1789760786.931865:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565526912:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760786.954520:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985246 gen 3 00000008:00010000:0.0:1789760786.954576:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760786.954584:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760786.954595:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760786.954703:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760786.954743:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565526912:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760786.955292:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17938, 17938] 00000080:00200000:1.0:1789760786.955310:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.955316:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.955332:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73478144 i_blocks 143536 00000020:00200000:1.0:1789760786.955337:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.955350:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.955385:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73478144, count: 4096 00000080:00200000:1.0:1789760786.955393:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73478144, bytes: 4096 00000080:00200000:1.0:1789760786.955404:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.955560:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.955563:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73478144, 73482240) 0 0 00000080:00200000:1.0:1789760786.955568:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.955586:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17939, 17939] 00020000:00200000:1.0:1789760786.955595:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73478144 chunk: [73478144, 73482240) 73482240, 4096 00020000:00200000:1.0:1789760786.955600:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.955621:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17939, 17939] 00000008:00010000:1.0:1789760786.955641:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.955667:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.955679:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73478144, 73482240) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.955697:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73478144=0x4613000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.955761:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.955764:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73482240 00000020:00010000:1.0:1789760786.955771:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.955775:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.955779:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.955782:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.955785:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.955793:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.955799:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.955822:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760786.955948:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73478144<>73482240 00000100:00100000:0.0:1789760786.956310:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760786.956336:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565527040:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760786.967329:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985247 gen 3 00000008:00010000:0.0:1789760786.967377:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760786.967476:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760786.967497:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760786.967975:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000080:00200000:1.0:1789760786.968002:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17939, 17939] 00000080:00200000:1.0:1789760786.968020:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.968028:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.968043:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73482240 i_blocks 143544 00000020:00200000:1.0:1789760786.968048:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.968058:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000100:00100000:0.0:1789760786.968062:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565527040:192.168.203.147@tcp:4:dd.0 00000080:00200008:1.0:1789760786.968091:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73482240, count: 4096 00000080:00200000:1.0:1789760786.968101:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73482240, bytes: 4096 00000080:00200000:1.0:1789760786.968110:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.968119:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.968121:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73482240, 73486336) 0 0 00000080:00200000:1.0:1789760786.968126:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.968143:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17940, 17940] 00020000:00200000:1.0:1789760786.968152:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73482240 chunk: [73482240, 73486336) 73486336, 4096 00020000:00200000:1.0:1789760786.968157:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.968176:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17940, 17940] 00000008:00010000:1.0:1789760786.968195:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.968202:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.968222:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73482240, 73486336) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.968236:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73482240=0x4614000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.968291:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.968294:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73486336 00000020:00010000:1.0:1789760786.968303:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.968305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.968307:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.968310:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.968312:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.968317:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.968323:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.968335:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760786.968440:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 73482240<>73486336 00000100:00100000:1.0:1789760786.968508:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565527168:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760786.971349:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760786.987212:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985248 gen 3 00000008:00010000:0.0:1789760786.987265:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760786.987273:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760786.987284:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760786.987309:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760786.987340:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565527168:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760786.987486:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17940, 17940] 00000080:00200000:1.0:1789760786.987503:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760786.987509:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760786.987523:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73486336 i_blocks 143552 00000020:00200000:1.0:1789760786.987526:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760786.987535:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760786.987865:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73486336, count: 4096 00000080:00200000:1.0:1789760786.987873:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73486336, bytes: 4096 00000080:00200000:1.0:1789760786.987885:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760786.987894:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760786.987896:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73486336, 73490432) 0 0 00000080:00200000:1.0:1789760786.987901:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760786.987918:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17941, 17941] 00020000:00200000:1.0:1789760786.987926:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73486336 chunk: [73486336, 73490432) 73490432, 4096 00020000:00200000:1.0:1789760786.987931:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760786.987952:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17941, 17941] 00000008:00010000:1.0:1789760786.987971:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760786.987977:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760786.987990:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73486336, 73490432) append 0 append lockpos 0 00000080:00200000:1.0:1789760786.988005:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73486336=0x4615000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760786.988066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760786.988069:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73490432 00000020:00010000:1.0:1789760786.988078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760786.988081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.988927:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760786.988937:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760786.988941:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760786.988952:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760786.988960:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760786.988985:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760786.989114:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 73486336<>73490432 00000100:00100000:0.0:1789760786.990359:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565527296:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.008598:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985249 gen 3 00000008:00010000:0.0:1789760787.008704:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760787.008714:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.008724:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.008737:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760787.008765:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565527296:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.008892:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17941, 17941] 00000080:00200000:1.0:1789760787.008910:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.008918:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.008934:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73490432 i_blocks 143560 00000020:00200000:1.0:1789760787.008938:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.008947:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.008986:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73490432, count: 4096 00000080:00200000:1.0:1789760787.008992:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73490432, bytes: 4096 00000080:00200000:1.0:1789760787.009007:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.009015:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.009017:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73490432, 73494528) 0 0 00000080:00200000:1.0:1789760787.009021:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.009038:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17942, 17942] 00020000:00200000:1.0:1789760787.009048:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73490432 chunk: [73490432, 73494528) 73494528, 4096 00020000:00200000:1.0:1789760787.009052:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.009073:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17942, 17942] 00000008:00010000:1.0:1789760787.009091:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.009097:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.009109:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73490432, 73494528) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.009125:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73490432=0x4616000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.009184:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.009187:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73494528 00000020:00010000:1.0:1789760787.009196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.009199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.009202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.009206:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.009209:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.009216:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.009223:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.009241:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760787.009365:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 73490432<>73494528 00000100:00100000:0.0:1789760787.010259:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565527424:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.029121:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985250 gen 3 00000008:00010000:0.0:1789760787.029173:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760787.029182:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.029193:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.029213:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760787.029242:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565527424:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.029352:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17942, 17942] 00000080:00200000:1.0:1789760787.029374:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.029380:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.029399:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73494528 i_blocks 143568 00000020:00200000:1.0:1789760787.029404:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.029414:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.029461:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73494528, count: 4096 00000080:00200000:1.0:1789760787.029469:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73494528, bytes: 4096 00000080:00200000:1.0:1789760787.029479:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.029489:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.029491:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73494528, 73498624) 0 0 00000080:00200000:1.0:1789760787.029496:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.029513:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17943, 17943] 00020000:00200000:1.0:1789760787.029523:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73494528 chunk: [73494528, 73498624) 73498624, 4096 00020000:00200000:1.0:1789760787.029527:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.029559:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17943, 17943] 00000008:00010000:1.0:1789760787.029585:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.029591:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.029602:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73494528, 73498624) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.029616:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73494528=0x4617000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.029715:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.029718:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73498624 00000020:00010000:1.0:1789760787.029726:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.029730:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.029733:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.029737:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.029739:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.029746:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.029753:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.029770:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760787.029908:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 73494528<>73498624 00000100:00100000:0.0:1789760787.030114:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565527552:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.040204:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985251 gen 3 00000008:00010000:0.0:1789760787.040254:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760787.040263:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.040273:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.040293:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760787.040323:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565527552:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.040427:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17943, 17943] 00000080:00200000:1.0:1789760787.040442:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.040458:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.040473:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73498624 i_blocks 143576 00000020:00200000:1.0:1789760787.040477:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.040487:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.040521:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73498624, count: 4096 00000080:00200000:1.0:1789760787.040529:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73498624, bytes: 4096 00000080:00200000:1.0:1789760787.040539:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.040560:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.040563:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73498624, 73502720) 0 0 00000080:00200000:1.0:1789760787.040567:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.040584:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17944, 17944] 00020000:00200000:1.0:1789760787.040592:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73498624 chunk: [73498624, 73502720) 73502720, 4096 00020000:00200000:1.0:1789760787.040597:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.040617:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17944, 17944] 00000008:00010000:1.0:1789760787.040634:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.040641:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.040688:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73498624, 73502720) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.040703:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73498624=0x4618000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.040758:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.040760:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73502720 00000020:00010000:1.0:1789760787.040768:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.040772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.040775:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.040778:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.040781:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.040787:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.040794:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.040809:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760787.040921:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 73498624<>73502720 00000100:00100000:0.0:1789760787.041153:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565527680:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.053057:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985252 gen 3 00000008:00010000:0.0:1789760787.053110:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760787.053118:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.053131:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.053150:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760787.053182:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565527680:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.053287:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17944, 17944] 00000080:00200000:1.0:1789760787.053305:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.053311:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.053328:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73502720 i_blocks 143584 00000020:00200000:1.0:1789760787.053332:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.053342:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.053377:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73502720, count: 4096 00000080:00200000:1.0:1789760787.053394:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73502720, bytes: 4096 00000080:00200000:1.0:1789760787.053404:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.053412:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.053415:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73502720, 73506816) 0 0 00000080:00200000:1.0:1789760787.053420:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.053438:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17945, 17945] 00020000:00200000:1.0:1789760787.053458:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73502720 chunk: [73502720, 73506816) 73506816, 4096 00020000:00200000:1.0:1789760787.053463:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.053484:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17945, 17945] 00000008:00010000:1.0:1789760787.053501:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.053507:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.053519:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73502720, 73506816) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.053533:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73502720=0x4619000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.053594:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.053597:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73506816 00000020:00010000:1.0:1789760787.053606:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.053609:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.053613:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.053616:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.053619:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.053628:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.053635:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.053696:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760787.053820:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 73502720<>73506816 00000100:00100000:0.0:1789760787.053986:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565527808:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.068955:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985253 gen 3 00000008:00010000:0.0:1789760787.069006:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760787.069015:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.069026:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.069051:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760787.069081:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565527808:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.069203:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17945, 17945] 00000080:00200000:1.0:1789760787.069220:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.069227:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.069241:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73506816 i_blocks 143592 00000020:00200000:1.0:1789760787.069245:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.069255:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.069287:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73506816, count: 4096 00000080:00200000:1.0:1789760787.069294:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73506816, bytes: 4096 00000080:00200000:1.0:1789760787.069304:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.069312:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.069315:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73506816, 73510912) 0 0 00000080:00200000:1.0:1789760787.069320:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.069338:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17946, 17946] 00020000:00200000:1.0:1789760787.069347:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73506816 chunk: [73506816, 73510912) 73510912, 4096 00020000:00200000:1.0:1789760787.069351:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.069373:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17946, 17946] 00000008:00010000:1.0:1789760787.069392:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.069397:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.069409:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73506816, 73510912) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.069424:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73506816=0x461a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.069493:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.069495:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73510912 00000020:00010000:1.0:1789760787.069503:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.069506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.069509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.069513:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.069516:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.069523:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.069529:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.069547:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760787.069717:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 73506816<>73510912 00000100:00100000:0.0:1789760787.070053:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565527936:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.078283:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985254 gen 3 00000008:00010000:0.0:1789760787.078614:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760787.078623:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.078635:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.078760:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760787.078792:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565527936:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.081031:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17946, 17946] 00000080:00200000:1.0:1789760787.081060:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.081068:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.081088:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73510912 i_blocks 143600 00000020:00200000:1.0:1789760787.081093:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.081110:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.081145:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73510912, count: 4096 00000080:00200000:1.0:1789760787.081153:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73510912, bytes: 4096 00000080:00200000:1.0:1789760787.081167:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.081177:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.081179:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73510912, 73515008) 0 0 00000080:00200000:1.0:1789760787.081184:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.081202:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17947, 17947] 00020000:00200000:1.0:1789760787.081211:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73510912 chunk: [73510912, 73515008) 73515008, 4096 00020000:00200000:1.0:1789760787.081215:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.081238:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17947, 17947] 00000008:00010000:1.0:1789760787.081258:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.081264:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.081276:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73510912, 73515008) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.081290:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73510912=0x461b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.081366:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.081369:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73515008 00000020:00010000:1.0:1789760787.081377:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.081381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.081383:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.081387:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.081390:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.081398:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.081405:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.081437:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760787.081681:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 73510912<>73515008 00000100:00100000:0.0:1789760787.081785:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565528064:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.091763:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985255 gen 3 00000008:00010000:0.0:1789760787.091823:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760787.091832:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.091843:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.091865:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760787.091895:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565528064:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.092313:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17947, 17947] 00000080:00200000:1.0:1789760787.092329:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.092337:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.092352:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73515008 i_blocks 143608 00000020:00200000:1.0:1789760787.092356:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.092366:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.092413:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73515008, count: 4096 00000080:00200000:1.0:1789760787.092421:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73515008, bytes: 4096 00000080:00200000:1.0:1789760787.092432:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.092440:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.092453:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73515008, 73519104) 0 0 00000080:00200000:1.0:1789760787.092457:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.092475:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17948, 17948] 00020000:00200000:1.0:1789760787.092494:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73515008 chunk: [73515008, 73519104) 73519104, 4096 00020000:00200000:1.0:1789760787.092500:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.092521:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17948, 17948] 00000008:00010000:1.0:1789760787.092537:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.092543:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.092555:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73515008, 73519104) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.092568:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73515008=0x461c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.092629:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.092632:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73519104 00000020:00010000:1.0:1789760787.092639:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.092690:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.092694:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.092698:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.092701:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.092708:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.092715:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.092734:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760787.092867:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 73515008<>73519104 00000100:00100000:0.0:1789760787.093208:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565528192:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.102145:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985256 gen 3 00000008:00010000:0.0:1789760787.102196:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760787.102205:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.102217:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.102325:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760787.102355:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565528192:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.102453:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17948, 17948] 00000080:00200000:1.0:1789760787.102469:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.102476:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.102566:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73519104 i_blocks 143616 00000020:00200000:1.0:1789760787.102571:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.102581:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.102618:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73519104, count: 4096 00000080:00200000:1.0:1789760787.102626:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73519104, bytes: 4096 00000080:00200000:1.0:1789760787.102636:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.102725:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.102728:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73519104, 73523200) 0 0 00000080:00200000:1.0:1789760787.102732:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.102751:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17949, 17949] 00020000:00200000:1.0:1789760787.102761:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73519104 chunk: [73519104, 73523200) 73523200, 4096 00020000:00200000:1.0:1789760787.102766:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.102786:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17949, 17949] 00000008:00010000:1.0:1789760787.102866:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.102872:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.102885:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73519104, 73523200) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.102899:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73519104=0x461d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.102954:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.102957:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73523200 00000020:00010000:1.0:1789760787.102965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.102969:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.102972:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.102975:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.102978:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.102985:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.102992:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.103009:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760787.103129:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 73519104<>73523200 00000100:00100000:0.0:1789760787.103193:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565528320:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.114134:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985257 gen 3 00000008:00010000:0.0:1789760787.114186:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760787.114195:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.114205:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.114590:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760787.114624:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565528320:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.114958:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17949, 17949] 00000080:00200000:1.0:1789760787.114975:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.114983:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.114999:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73523200 i_blocks 143624 00000020:00200000:1.0:1789760787.115003:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.115013:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.115046:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73523200, count: 4096 00000080:00200000:1.0:1789760787.115053:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73523200, bytes: 4096 00000080:00200000:1.0:1789760787.115064:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.115073:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.115076:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73523200, 73527296) 0 0 00000080:00200000:1.0:1789760787.115080:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.115098:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17950, 17950] 00020000:00200000:1.0:1789760787.115106:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73523200 chunk: [73523200, 73527296) 73527296, 4096 00020000:00200000:1.0:1789760787.115111:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.115130:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17950, 17950] 00000008:00010000:1.0:1789760787.115148:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.115154:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.115165:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73523200, 73527296) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.115180:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73523200=0x461e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.115298:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.115302:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73527296 00000020:00010000:1.0:1789760787.115312:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.115316:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.115319:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.115322:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.115326:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.115334:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.115341:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.115361:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760787.115619:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73523200<>73527296 00000100:00100000:0.0:1789760787.115754:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565528448:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.126067:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985258 gen 3 00000008:00010000:0.0:1789760787.126117:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760787.126125:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.126137:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.126213:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760787.126244:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565528448:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.129697:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17950, 17950] 00000080:00200000:1.0:1789760787.129726:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.129734:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.129754:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73527296 i_blocks 143632 00000020:00200000:1.0:1789760787.129760:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.129778:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.130560:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73527296, count: 4096 00000080:00200000:1.0:1789760787.130576:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73527296, bytes: 4096 00000080:00200000:1.0:1789760787.130593:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.130603:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.130605:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73527296, 73531392) 0 0 00000080:00200000:1.0:1789760787.130612:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.130631:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17951, 17951] 00020000:00200000:1.0:1789760787.130640:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73527296 chunk: [73527296, 73531392) 73531392, 4096 00020000:00200000:1.0:1789760787.130747:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.130771:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17951, 17951] 00000008:00010000:1.0:1789760787.130867:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.130875:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.130892:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73527296, 73531392) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.130906:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73527296=0x461f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.130981:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.130985:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73531392 00000020:00010000:1.0:1789760787.130993:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.130997:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.131002:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.131005:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.131008:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.131016:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.131024:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.131055:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760787.131185:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73527296<>73531392 00000100:00100000:0.0:1789760787.131271:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565528576:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.140160:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985259 gen 3 00000008:00010000:0.0:1789760787.140375:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760787.140384:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.140394:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.140424:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760787.140467:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565528576:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.140588:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17951, 17951] 00000080:00200000:1.0:1789760787.140606:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.140613:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.140630:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73531392 i_blocks 143640 00000020:00200000:1.0:1789760787.140634:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.140683:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.140718:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73531392, count: 4096 00000080:00200000:1.0:1789760787.140726:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73531392, bytes: 4096 00000080:00200000:1.0:1789760787.140735:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.141087:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.141090:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73531392, 73535488) 0 0 00000080:00200000:1.0:1789760787.141096:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.141206:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17952, 17952] 00020000:00200000:2.0:1789760787.141225:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73531392 chunk: [73531392, 73535488) 73535488, 4096 00020000:00200000:2.0:1789760787.141230:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.141250:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17952, 17952] 00000008:00010000:2.0:1789760787.141269:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.141274:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.141287:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73531392, 73535488) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.141300:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73531392=0x4620000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.141355:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.141358:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73535488 00000020:00010000:2.0:1789760787.141366:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.141370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.141373:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.141377:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.141381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.141389:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.141396:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.141461:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.141581:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 73531392<>73535488 00000100:00100000:2.0:1789760787.141636:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760787.141678:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565528704:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.149251:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985260 gen 3 00000008:00010000:2.0:1789760787.149302:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760787.149311:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.149322:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.149353:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760787.149393:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565528704:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760787.149687:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17952, 17952] 00000080:00200000:0.0:1789760787.149704:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760787.149709:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760787.149724:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73535488 i_blocks 143648 00000020:00200000:0.0:1789760787.149727:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760787.149738:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760787.149774:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73535488, count: 4096 00000080:00200000:0.0:1789760787.149783:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73535488, bytes: 4096 00000080:00200000:0.0:1789760787.149861:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760787.149870:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760787.149872:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73535488, 73539584) 0 0 00000080:00200000:0.0:1789760787.149877:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760787.149894:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17953, 17953] 00020000:00200000:0.0:1789760787.149903:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73535488 chunk: [73535488, 73539584) 73539584, 4096 00020000:00200000:0.0:1789760787.149908:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760787.149929:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17953, 17953] 00000008:00010000:0.0:1789760787.149946:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760787.149952:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760787.149964:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73535488, 73539584) append 0 append lockpos 0 00000080:00200000:0.0:1789760787.149980:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73535488=0x4621000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760787.150040:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760787.150043:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73539584 00000020:00010000:0.0:1789760787.150052:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760787.150056:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760787.150059:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760787.150062:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760787.150065:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760787.150072:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760787.150079:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760787.150096:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760787.150219:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73535488<>73539584 00000100:00100000:1.0:1789760787.150295:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565528832:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.158963:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985261 gen 3 00000008:00010000:0.0:1789760787.159015:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760787.159024:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.159035:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.159062:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760787.159093:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565528832:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.159213:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17953, 17953] 00000080:00200000:2.0:1789760787.159229:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.159235:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.159251:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73539584 i_blocks 143656 00000020:00200000:2.0:1789760787.159255:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.159266:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.159302:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73539584, count: 4096 00000080:00200000:2.0:1789760787.159310:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73539584, bytes: 4096 00000080:00200000:2.0:1789760787.159320:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.159330:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.159333:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73539584, 73543680) 0 0 00000080:00200000:2.0:1789760787.159338:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.159356:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17954, 17954] 00020000:00200000:2.0:1789760787.159366:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73539584 chunk: [73539584, 73543680) 73543680, 4096 00020000:00200000:2.0:1789760787.159371:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.159404:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17954, 17954] 00000008:00010000:2.0:1789760787.159424:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.159430:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.159453:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73539584, 73543680) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.159468:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73539584=0x4622000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.159529:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.159532:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73543680 00000020:00010000:2.0:1789760787.159540:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.159543:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.159547:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.159550:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.159553:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.159561:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.159568:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.159584:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.159756:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 73539584<>73543680 00000100:00100000:3.0:1789760787.159939:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565528960:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.171714:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985262 gen 3 00000008:00010000:3.0:1789760787.171777:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760787.171786:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.171805:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.171836:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760787.171866:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565528960:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.171898:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17954, 17954] 00000080:00200000:2.0:1789760787.171917:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.171923:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.171938:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73543680 i_blocks 143664 00000020:00200000:2.0:1789760787.171942:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.171952:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.172041:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73543680, count: 4096 00000080:00200000:2.0:1789760787.172050:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73543680, bytes: 4096 00000080:00200000:2.0:1789760787.172059:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.172067:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.172069:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73543680, 73547776) 0 0 00000080:00200000:2.0:1789760787.172074:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.172091:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17955, 17955] 00020000:00200000:2.0:1789760787.172100:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73543680 chunk: [73543680, 73547776) 73547776, 4096 00020000:00200000:2.0:1789760787.172104:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.172124:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17955, 17955] 00000008:00010000:2.0:1789760787.172141:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.172148:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.172159:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73543680, 73547776) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.172171:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73543680=0x4623000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.172230:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.172234:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73547776 00000020:00010000:2.0:1789760787.172241:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.172245:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.172249:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.172252:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.172255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.172264:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.172271:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.172286:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760787.172406:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 73543680<>73547776 00000100:00100000:3.0:1789760787.173778:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565529088:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.184112:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985263 gen 3 00000008:00010000:3.0:1789760787.184163:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760787.184171:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.184182:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.184205:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760787.184237:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565529088:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.184356:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17955, 17955] 00000080:00200000:2.0:1789760787.184373:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.184388:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.184405:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73547776 i_blocks 143672 00000020:00200000:2.0:1789760787.184409:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.184421:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.184469:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73547776, count: 4096 00000080:00200000:2.0:1789760787.184476:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73547776, bytes: 4096 00000080:00200000:2.0:1789760787.184486:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.184495:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.184498:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73547776, 73551872) 0 0 00000080:00200000:2.0:1789760787.184502:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.184521:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17956, 17956] 00020000:00200000:2.0:1789760787.184530:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73547776 chunk: [73547776, 73551872) 73551872, 4096 00020000:00200000:2.0:1789760787.184535:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.184555:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17956, 17956] 00000008:00010000:2.0:1789760787.184576:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.184582:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.184594:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73547776, 73551872) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.184608:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73547776=0x4624000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.184703:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.184706:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73551872 00000020:00010000:2.0:1789760787.184714:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.184718:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.184721:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.184725:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.184728:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.184736:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.184742:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.184758:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760787.184890:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 73547776<>73551872 00000100:00100000:3.0:1789760787.185051:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565529216:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.195542:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985264 gen 3 00000008:00010000:3.0:1789760787.195593:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760787.195601:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.195612:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.195636:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760787.195705:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565529216:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.195934:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17956, 17956] 00000080:00200000:2.0:1789760787.195953:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.195960:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.195979:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73551872 i_blocks 143680 00000020:00200000:2.0:1789760787.195984:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.195995:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.196031:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73551872, count: 4096 00000080:00200000:2.0:1789760787.196038:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73551872, bytes: 4096 00000080:00200000:2.0:1789760787.196048:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.196057:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.196060:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73551872, 73555968) 0 0 00000080:00200000:2.0:1789760787.196064:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.196081:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17957, 17957] 00020000:00200000:2.0:1789760787.196090:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73551872 chunk: [73551872, 73555968) 73555968, 4096 00020000:00200000:2.0:1789760787.196094:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.196115:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17957, 17957] 00000008:00010000:2.0:1789760787.196135:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.196140:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.196152:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73551872, 73555968) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.196168:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73551872=0x4625000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.196228:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.196231:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73555968 00000020:00010000:2.0:1789760787.196239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.196243:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.196247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.196250:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.196254:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.196261:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.196267:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.196285:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.196422:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 73551872<>73555968 00000100:00100000:3.0:1789760787.197961:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565529344:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.211588:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985265 gen 3 00000008:00010000:3.0:1789760787.211639:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760787.211691:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.211701:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.211724:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760787.211752:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565529344:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.212049:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17957, 17957] 00000080:00200000:2.0:1789760787.212066:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.212072:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.212088:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73555968 i_blocks 143688 00000020:00200000:2.0:1789760787.212092:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.212102:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.212133:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73555968, count: 4096 00000080:00200000:2.0:1789760787.212140:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73555968, bytes: 4096 00000080:00200000:2.0:1789760787.212150:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.212158:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.212161:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73555968, 73560064) 0 0 00000080:00200000:2.0:1789760787.212165:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.212180:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17958, 17958] 00020000:00200000:2.0:1789760787.212188:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73555968 chunk: [73555968, 73560064) 73560064, 4096 00020000:00200000:2.0:1789760787.212193:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.212210:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17958, 17958] 00000008:00010000:2.0:1789760787.212225:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.212231:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.212241:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73555968, 73560064) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.212255:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73555968=0x4626000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.212306:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.212308:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73560064 00000020:00010000:2.0:1789760787.212316:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.212320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.212323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.212326:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.212329:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.212336:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.212342:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.212357:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760787.212483:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 73555968<>73560064 00000100:00100000:3.0:1789760787.212581:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565529472:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.222854:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985266 gen 3 00000008:00010000:3.0:1789760787.222906:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760787.222914:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.222924:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.222944:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760787.222968:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565529472:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.224533:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17958, 17958] 00000080:00200000:2.0:1789760787.224559:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.224566:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.224584:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73560064 i_blocks 143696 00000020:00200000:2.0:1789760787.224589:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.224603:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.224638:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73560064, count: 4096 00000080:00200000:2.0:1789760787.224681:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73560064, bytes: 4096 00000080:00200000:2.0:1789760787.224696:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.224704:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.224707:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73560064, 73564160) 0 0 00000080:00200000:2.0:1789760787.224710:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.224727:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17959, 17959] 00020000:00200000:2.0:1789760787.224736:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73560064 chunk: [73560064, 73564160) 73564160, 4096 00020000:00200000:2.0:1789760787.224741:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.224764:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17959, 17959] 00000008:00010000:2.0:1789760787.224783:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.224789:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.224801:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73560064, 73564160) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.224815:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73560064=0x4627000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.224880:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.224883:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73564160 00000020:00010000:2.0:1789760787.224892:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.224895:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.224898:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.224901:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.224904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.224912:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.224919:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.224940:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760787.225066:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 73560064<>73564160 00000100:00100000:3.0:1789760787.225287:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565529600:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.237115:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985267 gen 3 00000008:00010000:3.0:1789760787.237166:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760787.237175:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.237186:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.237210:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760787.237250:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565529600:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.237394:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17959, 17959] 00000080:00200000:2.0:1789760787.237412:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.237417:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.237432:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73564160 i_blocks 143704 00000020:00200000:2.0:1789760787.237441:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.237459:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.237495:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73564160, count: 4096 00000080:00200000:2.0:1789760787.237503:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73564160, bytes: 4096 00000080:00200000:2.0:1789760787.237512:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.237520:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.237523:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73564160, 73568256) 0 0 00000080:00200000:2.0:1789760787.237527:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.237544:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17960, 17960] 00020000:00200000:2.0:1789760787.237553:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73564160 chunk: [73564160, 73568256) 73568256, 4096 00020000:00200000:2.0:1789760787.237558:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.237579:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17960, 17960] 00000008:00010000:2.0:1789760787.237597:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.237604:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.237616:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73564160, 73568256) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.237630:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73564160=0x4628000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.237722:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.237725:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73568256 00000020:00010000:2.0:1789760787.237734:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.237737:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.237740:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.237744:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.237746:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.237755:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.237761:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.237776:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.237899:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 73564160<>73568256 00000100:00100000:3.0:1789760787.238536:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565529728:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.254228:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985268 gen 3 00000008:00010000:3.0:1789760787.254278:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760787.254287:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.254298:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.254331:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760787.254362:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565529728:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.254567:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17960, 17960] 00000080:00200000:2.0:1789760787.254584:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.254590:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.254606:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73568256 i_blocks 143712 00000020:00200000:2.0:1789760787.254609:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.254619:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.254688:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73568256, count: 4096 00000080:00200000:2.0:1789760787.254695:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73568256, bytes: 4096 00000080:00200000:2.0:1789760787.254707:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.254715:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.254717:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73568256, 73572352) 0 0 00000080:00200000:2.0:1789760787.254722:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.254739:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17961, 17961] 00020000:00200000:2.0:1789760787.254748:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73568256 chunk: [73568256, 73572352) 73572352, 4096 00020000:00200000:2.0:1789760787.254753:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.254773:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17961, 17961] 00000008:00010000:2.0:1789760787.254791:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.254797:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.254809:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73568256, 73572352) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.254827:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73568256=0x4629000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.254884:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.254887:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73572352 00000020:00010000:2.0:1789760787.254896:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.254900:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.254902:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.254906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.254909:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.254917:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.254923:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.254940:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760787.255060:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 73568256<>73572352 00000100:00100000:3.0:1789760787.256518:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565529856:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.271281:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985269 gen 3 00000008:00010000:3.0:1789760787.271333:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760787.271341:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.271352:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.271375:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760787.271475:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565529856:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.273553:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17961, 17961] 00000080:00200000:2.0:1789760787.273580:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.273588:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.273607:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73572352 i_blocks 143720 00000020:00200000:2.0:1789760787.273612:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.273628:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.273696:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73572352, count: 4096 00000080:00200000:2.0:1789760787.273706:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73572352, bytes: 4096 00000080:00200000:2.0:1789760787.273719:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.273729:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.273731:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73572352, 73576448) 0 0 00000080:00200000:2.0:1789760787.273735:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.273751:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17962, 17962] 00020000:00200000:2.0:1789760787.273760:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73572352 chunk: [73572352, 73576448) 73576448, 4096 00020000:00200000:2.0:1789760787.273764:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.273787:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17962, 17962] 00000008:00010000:2.0:1789760787.273805:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.273811:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.273824:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73572352, 73576448) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.273840:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73572352=0x462a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.273911:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.273914:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73576448 00000020:00010000:2.0:1789760787.273922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.273926:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.273930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.273933:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.273936:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.273945:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.273951:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.273971:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760787.274109:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 73572352<>73576448 00000100:00100000:3.0:1789760787.274630:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565529984:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.290053:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985270 gen 3 00000008:00010000:3.0:1789760787.290107:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:3.0:1789760787.290115:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.290127:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.290230:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:3.0:1789760787.290259:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565529984:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.290348:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17962, 17962] 00000080:00200000:2.0:1789760787.290366:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.290371:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.290385:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73576448 i_blocks 143728 00000020:00200000:2.0:1789760787.290389:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.290400:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.290435:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73576448, count: 4096 00000080:00200000:2.0:1789760787.290452:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73576448, bytes: 4096 00000080:00200000:2.0:1789760787.290467:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.290476:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.290479:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73576448, 73580544) 0 0 00000080:00200000:2.0:1789760787.290483:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.290508:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17963, 17963] 00020000:00200000:2.0:1789760787.290517:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73576448 chunk: [73576448, 73580544) 73580544, 4096 00020000:00200000:2.0:1789760787.290522:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.290543:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17963, 17963] 00000008:00010000:2.0:1789760787.290559:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.290565:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.290577:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73576448, 73580544) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.290591:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73576448=0x462b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.294317:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.294330:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73580544 00000020:00010000:2.0:1789760787.294347:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.294352:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.294357:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.294362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.294365:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.294430:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.294441:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.294482:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.294974:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 73576448<>73580544 00000100:00100000:2.0:1789760787.295035:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760787.295062:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565530112:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.310707:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985271 gen 3 00000008:00010000:2.0:1789760787.310759:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760787.310768:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.310779:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.310854:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760787.310884:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565530112:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.311507:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17963, 17963] 00000080:00200000:3.0:1789760787.311526:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.311598:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.311746:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73580544 i_blocks 143736 00000020:00200000:2.0:1789760787.311752:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.311765:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.311801:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73580544, count: 4096 00000080:00200000:2.0:1789760787.311809:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73580544, bytes: 4096 00000080:00200000:2.0:1789760787.311821:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.311830:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.311832:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73580544, 73584640) 0 0 00000080:00200000:2.0:1789760787.311836:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.311855:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17964, 17964] 00020000:00200000:2.0:1789760787.311864:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73580544 chunk: [73580544, 73584640) 73584640, 4096 00020000:00200000:2.0:1789760787.311869:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.311888:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17964, 17964] 00000008:00010000:2.0:1789760787.311907:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.311913:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.311924:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73580544, 73584640) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.311937:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73580544=0x462c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.311992:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.311995:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73584640 00000020:00010000:2.0:1789760787.312002:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.312006:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.312009:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.312013:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.312016:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.312024:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.312031:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.312081:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.313696:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 73580544<>73584640 00000100:00100000:3.0:1789760787.315320:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565530240:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.330474:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985272 gen 3 00000008:00010000:3.0:1789760787.330590:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760787.330599:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.330610:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.330639:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760787.330755:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565530240:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760787.330773:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17964, 17964] 00000080:00200000:0.0:1789760787.330791:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760787.330797:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760787.330812:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73584640 i_blocks 143744 00000020:00200000:0.0:1789760787.330817:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760787.330829:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760787.330863:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73584640, count: 4096 00000080:00200000:0.0:1789760787.330872:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73584640, bytes: 4096 00000080:00200000:0.0:1789760787.330883:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760787.330893:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760787.330896:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73584640, 73588736) 0 0 00000080:00200000:0.0:1789760787.330903:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760787.330921:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17965, 17965] 00020000:00200000:0.0:1789760787.330930:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73584640 chunk: [73584640, 73588736) 73588736, 4096 00020000:00200000:0.0:1789760787.330935:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760787.330956:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17965, 17965] 00000008:00010000:0.0:1789760787.330975:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760787.330981:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760787.330994:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73584640, 73588736) append 0 append lockpos 0 00000080:00200000:0.0:1789760787.331007:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73584640=0x462d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760787.331064:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760787.331067:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73588736 00000020:00010000:0.0:1789760787.331076:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760787.331080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760787.331083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760787.331086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84418a6c8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760787.331089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a6c8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760787.331097:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760787.331105:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760787.332413:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760787.332584:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73584640<>73588736 00000100:00100000:1.0:1789760787.332682:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565530368:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.342983:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985273 gen 3 00000008:00010000:0.0:1789760787.343037:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760787.343045:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.343057:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.343147:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760787.343179:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565530368:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.343305:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17965, 17965] 00000080:00200000:3.0:1789760787.343322:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.343328:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.343344:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73588736 i_blocks 143752 00000020:00200000:3.0:1789760787.343348:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.343359:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.343439:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73588736, count: 4096 00000080:00200000:3.0:1789760787.343459:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73588736, bytes: 4096 00000080:00200000:3.0:1789760787.343471:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.343479:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.343482:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73588736, 73592832) 0 0 00000080:00200000:3.0:1789760787.346197:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.346234:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17966, 17966] 00020000:00200000:3.0:1789760787.346245:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73588736 chunk: [73588736, 73592832) 73592832, 4096 00020000:00200000:3.0:1789760787.346250:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.346275:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17966, 17966] 00000008:00010000:3.0:1789760787.346298:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.346304:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.346317:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73588736, 73592832) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.346333:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73588736=0x462e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.346405:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.346408:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73592832 00000020:00010000:3.0:1789760787.346415:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.346419:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.346422:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.346425:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.346428:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.346439:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.346458:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.346490:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760787.347290:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 73588736<>73592832 00000100:00100000:3.0:1789760787.347360:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565530496:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.356515:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985274 gen 3 00000008:00010000:3.0:1789760787.356566:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:3.0:1789760787.356575:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.356587:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.356922:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:3.0:1789760787.356950:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565530496:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.357078:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17966, 17966] 00000080:00200000:2.0:1789760787.357094:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.357100:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.357117:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73592832 i_blocks 143760 00000020:00200000:2.0:1789760787.357121:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.357131:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.357169:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73592832, count: 4096 00000080:00200000:2.0:1789760787.357177:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73592832, bytes: 4096 00000080:00200000:2.0:1789760787.357188:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.357197:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.357200:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73592832, 73596928) 0 0 00000080:00200000:2.0:1789760787.357204:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.357221:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17967, 17967] 00020000:00200000:2.0:1789760787.357229:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73592832 chunk: [73592832, 73596928) 73596928, 4096 00020000:00200000:2.0:1789760787.357234:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.357253:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17967, 17967] 00000008:00010000:2.0:1789760787.357271:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.357277:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.357290:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73592832, 73596928) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.357304:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73592832=0x462f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.357367:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.357369:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73596928 00000020:00010000:2.0:1789760787.357378:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.357455:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.357459:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.357463:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.357466:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.357475:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.357482:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.357499:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760787.357620:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 73592832<>73596928 00000100:00100000:3.0:1789760787.357695:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565530624:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.365702:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985275 gen 3 00000008:00010000:3.0:1789760787.365754:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760787.365770:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.365781:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.365808:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760787.365838:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565530624:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.366221:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17967, 17967] 00000080:00200000:2.0:1789760787.366238:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.366244:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.366261:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73596928 i_blocks 143768 00000020:00200000:2.0:1789760787.366265:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.366275:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.366305:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73596928, count: 4096 00000080:00200000:2.0:1789760787.366312:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73596928, bytes: 4096 00000080:00200000:2.0:1789760787.366322:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.366331:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.366334:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73596928, 73601024) 0 0 00000080:00200000:2.0:1789760787.366338:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.366354:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17968, 17968] 00020000:00200000:2.0:1789760787.366362:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73596928 chunk: [73596928, 73601024) 73601024, 4096 00020000:00200000:2.0:1789760787.366367:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.366398:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17968, 17968] 00000008:00010000:2.0:1789760787.366416:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.366421:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.366434:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73596928, 73601024) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.366458:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73596928=0x4630000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.366516:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.366519:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73601024 00000020:00010000:2.0:1789760787.366527:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.366531:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.366533:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.366536:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.366540:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.366553:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.366559:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.366575:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.366729:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 73596928<>73601024 00000100:00100000:3.0:1789760787.367010:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565530752:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.375697:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985276 gen 3 00000008:00010000:3.0:1789760787.375742:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760787.375750:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.375761:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.375834:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760787.375865:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565530752:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.375989:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17968, 17968] 00000080:00200000:2.0:1789760787.376005:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.376011:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.376025:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73601024 i_blocks 143776 00000020:00200000:2.0:1789760787.376028:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.376038:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.376069:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73601024, count: 4096 00000080:00200000:2.0:1789760787.376076:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73601024, bytes: 4096 00000080:00200000:2.0:1789760787.376088:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.376097:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.376099:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73601024, 73605120) 0 0 00000080:00200000:2.0:1789760787.376104:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.376120:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17969, 17969] 00020000:00200000:2.0:1789760787.376128:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73601024 chunk: [73601024, 73605120) 73605120, 4096 00020000:00200000:2.0:1789760787.376133:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.376150:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17969, 17969] 00000008:00010000:2.0:1789760787.376167:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.376172:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.376183:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73601024, 73605120) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.376196:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73601024=0x4631000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.376246:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.376249:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73605120 00000020:00010000:2.0:1789760787.376256:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.376259:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.376262:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.376266:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.376269:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.376276:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.376282:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.376296:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189000 00000008:00100000:2.0:1789760787.377534:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 73601024<>73605120 00000100:00100000:3.0:1789760787.377622:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565530880:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.384825:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985277 gen 3 00000008:00010000:2.0:1789760787.384880:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189000 00000020:00200000:2.0:1789760787.384890:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.384901:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.384922:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189000 00000100:00100000:2.0:1789760787.384950:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565530880:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.384950:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17969, 17969] 00000080:00200000:3.0:1789760787.384969:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.384975:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.384990:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73605120 i_blocks 143784 00000020:00200000:3.0:1789760787.384993:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.385003:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.385095:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73605120, count: 4096 00000080:00200000:2.0:1789760787.385104:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73605120, bytes: 4096 00000080:00200000:2.0:1789760787.385114:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.385123:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.385126:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73605120, 73609216) 0 0 00000080:00200000:2.0:1789760787.385131:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.385150:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17970, 17970] 00020000:00200000:2.0:1789760787.385159:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73605120 chunk: [73605120, 73609216) 73609216, 4096 00020000:00200000:2.0:1789760787.385165:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.385300:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17970, 17970] 00000008:00010000:2.0:1789760787.385318:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.385323:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.385336:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73605120, 73609216) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.385349:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73605120=0x4632000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.385399:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.385401:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73609216 00000020:00010000:2.0:1789760787.385409:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.385413:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.385416:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.385420:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.385423:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.385430:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.385437:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.385473:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.385713:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 73605120<>73609216 00000100:00100000:2.0:1789760787.385751:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760787.385767:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565531008:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.397765:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985278 gen 3 00000008:00010000:2.0:1789760787.397816:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760787.397824:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.397835:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.397864:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760787.397894:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565531008:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.398021:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17970, 17970] 00000080:00200000:3.0:1789760787.398070:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.398076:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.398091:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73609216 i_blocks 143792 00000020:00200000:3.0:1789760787.398095:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.398103:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.398147:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73609216, count: 4096 00000080:00200000:3.0:1789760787.398155:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73609216, bytes: 4096 00000080:00200000:3.0:1789760787.398169:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.398178:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.398180:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73609216, 73613312) 0 0 00000080:00200000:3.0:1789760787.398184:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.398200:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17971, 17971] 00020000:00200000:3.0:1789760787.398209:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73609216 chunk: [73609216, 73613312) 73613312, 4096 00020000:00200000:3.0:1789760787.398214:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.398245:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17971, 17971] 00000008:00010000:3.0:1789760787.398261:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.398267:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.398278:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73609216, 73613312) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.398291:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73609216=0x4633000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.398343:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.398345:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73613312 00000020:00010000:3.0:1789760787.398352:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.398356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.398359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.398362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.398365:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.398373:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.398379:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.398394:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760787.398524:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 73609216<>73613312 00000100:00100000:3.0:1789760787.398566:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760787.398582:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565531136:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760787.409432:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985279 gen 3 00000008:00010000:3.0:1789760787.409497:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:3.0:1789760787.409505:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760787.409516:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760787.409543:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:3.0:1789760787.409574:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565531136:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760787.409872:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17971, 17971] 00000080:00200000:0.0:1789760787.409889:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760787.409895:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760787.409911:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73613312 i_blocks 143800 00000020:00200000:0.0:1789760787.409915:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760787.409926:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760787.409961:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73613312, count: 4096 00000080:00200000:0.0:1789760787.409968:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73613312, bytes: 4096 00000080:00200000:0.0:1789760787.409979:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760787.409987:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760787.409990:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73613312, 73617408) 0 0 00000080:00200000:0.0:1789760787.409994:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760787.410010:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17972, 17972] 00020000:00200000:0.0:1789760787.410019:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73613312 chunk: [73613312, 73617408) 73617408, 4096 00020000:00200000:0.0:1789760787.410024:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760787.410044:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17972, 17972] 00000008:00010000:0.0:1789760787.410062:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760787.410068:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760787.410079:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73613312, 73617408) append 0 append lockpos 0 00000080:00200000:0.0:1789760787.410091:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73613312=0x4634000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760787.410150:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760787.410153:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73617408 00000020:00010000:0.0:1789760787.410161:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:0.0:1789760787.410164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760787.410167:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760787.410170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760787.410173:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760787.410181:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760787.410188:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760787.410205:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760787.410331:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73613312<>73617408 00000100:00100000:1.0:1789760787.410422:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565531264:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.422554:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985280 gen 3 00000008:00010000:0.0:1789760787.422610:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760787.422619:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.422631:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.422747:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760787.422780:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565531264:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.423481:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17972, 17972] 00000080:00200000:3.0:1789760787.423506:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.423512:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.423527:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73617408 i_blocks 143808 00000020:00200000:3.0:1789760787.423532:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.423544:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.423580:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73617408, count: 4096 00000080:00200000:3.0:1789760787.423586:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73617408, bytes: 4096 00000080:00200000:3.0:1789760787.423599:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.423608:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.423610:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73617408, 73621504) 0 0 00000080:00200000:3.0:1789760787.423615:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.423632:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17973, 17973] 00020000:00200000:3.0:1789760787.423641:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73617408 chunk: [73617408, 73621504) 73621504, 4096 00020000:00200000:3.0:1789760787.423678:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.423699:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17973, 17973] 00000008:00010000:3.0:1789760787.423717:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.423723:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.423735:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73617408, 73621504) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.423747:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73617408=0x4635000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.423815:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.423817:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73621504 00000020:00010000:3.0:1789760787.423824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.423828:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.423831:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.423835:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.423838:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.423846:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.423852:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.423873:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760787.423989:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 73617408<>73621504 00000100:00100000:2.0:1789760787.424254:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565531392:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.437042:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985281 gen 3 00000008:00010000:2.0:1789760787.437097:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760787.437105:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.437116:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.437136:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:2.0:1789760787.437165:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565531392:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.437970:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17973, 17973] 00000080:00200000:3.0:1789760787.437990:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.437996:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.438013:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73621504 i_blocks 143816 00000020:00200000:3.0:1789760787.438017:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.438028:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.438062:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73621504, count: 4096 00000080:00200000:3.0:1789760787.438069:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73621504, bytes: 4096 00000080:00200000:3.0:1789760787.438080:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.438090:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.438092:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73621504, 73625600) 0 0 00000080:00200000:3.0:1789760787.438097:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.438115:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17974, 17974] 00020000:00200000:3.0:1789760787.438124:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73621504 chunk: [73621504, 73625600) 73625600, 4096 00020000:00200000:3.0:1789760787.438129:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.438149:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17974, 17974] 00000008:00010000:3.0:1789760787.438167:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.438173:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.438186:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73621504, 73625600) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.438200:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73621504=0x4636000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.438264:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.438268:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73625600 00000020:00010000:3.0:1789760787.438277:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.438280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.438283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.438288:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.438291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.438300:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.438306:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.438327:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760787.438470:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 73621504<>73625600 00000100:00100000:2.0:1789760787.438664:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760787.438688:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565531520:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.455991:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985282 gen 3 00000008:00010000:2.0:1789760787.456042:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760787.456051:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.456063:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.456084:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760787.456114:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565531520:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.456217:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17974, 17974] 00000080:00200000:3.0:1789760787.456234:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.456240:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.456255:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73625600 i_blocks 143824 00000020:00200000:3.0:1789760787.456258:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.456269:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.456304:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73625600, count: 4096 00000080:00200000:3.0:1789760787.456312:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73625600, bytes: 4096 00000080:00200000:3.0:1789760787.456321:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.456330:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.456333:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73625600, 73629696) 0 0 00000080:00200000:3.0:1789760787.456338:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.456357:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17975, 17975] 00020000:00200000:3.0:1789760787.456366:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73625600 chunk: [73625600, 73629696) 73629696, 4096 00020000:00200000:3.0:1789760787.456371:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.456390:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17975, 17975] 00000008:00010000:3.0:1789760787.456408:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.456414:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.456424:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73625600, 73629696) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.456439:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73625600=0x4637000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.456505:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.456508:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73629696 00000020:00010000:3.0:1789760787.456516:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.456520:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.456523:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.456527:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.456530:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.456539:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.456546:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.456562:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760787.456731:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 73625600<>73629696 00000100:00100000:2.0:1789760787.456793:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565531648:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.467209:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985283 gen 3 00000008:00010000:2.0:1789760787.467262:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760787.467271:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.467283:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.467304:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760787.467334:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565531648:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.467891:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17975, 17975] 00000080:00200000:3.0:1789760787.467909:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.467915:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.467930:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73629696 i_blocks 143832 00000020:00200000:3.0:1789760787.467934:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.467945:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.467978:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73629696, count: 4096 00000080:00200000:3.0:1789760787.467986:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73629696, bytes: 4096 00000080:00200000:3.0:1789760787.467995:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.468004:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.468007:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73629696, 73633792) 0 0 00000080:00200000:3.0:1789760787.468011:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.468028:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17976, 17976] 00020000:00200000:3.0:1789760787.468037:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73629696 chunk: [73629696, 73633792) 73633792, 4096 00020000:00200000:3.0:1789760787.468042:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.468063:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17976, 17976] 00000008:00010000:3.0:1789760787.468083:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.468089:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.468102:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73629696, 73633792) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.468115:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73629696=0x4638000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.468173:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.468176:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73633792 00000020:00010000:3.0:1789760787.468184:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.468188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.468191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.468196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.468199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.468207:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.468214:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.468230:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760787.468358:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66d80 - object 0x240000400:5 offset 73629696<>73633792 00000100:00100000:2.0:1789760787.469169:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565531776:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.478335:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985284 gen 3 00000008:00010000:2.0:1789760787.478482:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760787.478491:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.478502:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.478522:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760787.478551:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565531776:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.478665:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17976, 17976] 00000080:00200000:3.0:1789760787.478681:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.478687:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.478700:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73633792 i_blocks 143840 00000020:00200000:3.0:1789760787.478704:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.478713:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.478746:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73633792, count: 4096 00000080:00200000:3.0:1789760787.478753:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73633792, bytes: 4096 00000080:00200000:3.0:1789760787.478772:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.478781:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.478784:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73633792, 73637888) 0 0 00000080:00200000:3.0:1789760787.478789:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.478805:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17977, 17977] 00020000:00200000:3.0:1789760787.478812:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73633792 chunk: [73633792, 73637888) 73637888, 4096 00020000:00200000:3.0:1789760787.478817:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.478835:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17977, 17977] 00000008:00010000:3.0:1789760787.478852:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.478857:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.478868:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73633792, 73637888) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.478882:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73633792=0x4639000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.478933:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.478936:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73637888 00000020:00010000:3.0:1789760787.478943:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.478947:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.478950:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.478954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.478957:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.478965:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.478971:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.478986:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760787.479109:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 73633792<>73637888 00000100:00100000:2.0:1789760787.479282:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565531904:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.489271:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985285 gen 3 00000008:00010000:2.0:1789760787.489322:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760787.489330:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.489342:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.489365:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760787.489401:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565531904:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.489581:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17977, 17977] 00000080:00200000:3.0:1789760787.489598:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.489605:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.489619:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73637888 i_blocks 143848 00000020:00200000:3.0:1789760787.489623:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.489633:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.489696:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73637888, count: 4096 00000080:00200000:3.0:1789760787.489704:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73637888, bytes: 4096 00000080:00200000:3.0:1789760787.489714:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.489722:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.489725:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73637888, 73641984) 0 0 00000080:00200000:3.0:1789760787.489729:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.489745:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17978, 17978] 00020000:00200000:3.0:1789760787.489755:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73637888 chunk: [73637888, 73641984) 73641984, 4096 00020000:00200000:3.0:1789760787.489823:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.489846:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17978, 17978] 00000008:00010000:3.0:1789760787.489863:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.489869:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.489880:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73637888, 73641984) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.489892:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73637888=0x463a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.489947:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.489951:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73641984 00000020:00010000:3.0:1789760787.489959:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.489962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.489965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.489969:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.489972:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.489980:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.489987:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.490005:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760787.490122:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 73637888<>73641984 00000100:00100000:2.0:1789760787.491245:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565532032:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.506024:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985286 gen 3 00000008:00010000:2.0:1789760787.506072:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760787.506088:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.506101:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.506231:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760787.506261:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565532032:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.507132:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17978, 17978] 00000080:00200000:3.0:1789760787.507148:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.507154:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.507359:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73641984 i_blocks 143856 00000020:00200000:2.0:1789760787.507364:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.507376:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.507419:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73641984, count: 4096 00000080:00200000:2.0:1789760787.507426:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73641984, bytes: 4096 00000080:00200000:2.0:1789760787.507448:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.507457:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.507460:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73641984, 73646080) 0 0 00000080:00200000:2.0:1789760787.507465:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.507484:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17979, 17979] 00020000:00200000:2.0:1789760787.507493:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73641984 chunk: [73641984, 73646080) 73646080, 4096 00020000:00200000:2.0:1789760787.507498:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.507521:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17979, 17979] 00000008:00010000:2.0:1789760787.507540:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.507546:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.507559:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73641984, 73646080) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.507575:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73641984=0x463b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.507640:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.507885:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73646080 00000020:00010000:2.0:1789760787.507895:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.507899:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.507902:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.507906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0218 (ffff93f84418ac68/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.507910:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.507918:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.507924:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.507976:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.508339:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 73641984<>73646080 00000100:00100000:2.0:1789760787.508400:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565532160:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.521617:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985287 gen 3 00000008:00010000:2.0:1789760787.521714:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760787.521724:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.521735:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.521768:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760787.521798:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565532160:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.522052:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17979, 17979] 00000080:00200000:3.0:1789760787.522069:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.522075:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.522090:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73646080 i_blocks 143864 00000020:00200000:3.0:1789760787.522094:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.522104:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.522168:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73646080, count: 4096 00000080:00200000:3.0:1789760787.522176:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73646080, bytes: 4096 00000080:00200000:3.0:1789760787.522188:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.522197:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.522199:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73646080, 73650176) 0 0 00000080:00200000:3.0:1789760787.522204:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.522221:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17980, 17980] 00020000:00200000:3.0:1789760787.522230:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73646080 chunk: [73646080, 73650176) 73650176, 4096 00020000:00200000:3.0:1789760787.522235:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.522254:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17980, 17980] 00000008:00010000:3.0:1789760787.522271:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.522278:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.522289:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73646080, 73650176) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.522303:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73646080=0x463c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.522359:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.522362:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73650176 00000020:00010000:3.0:1789760787.522371:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.522375:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.523003:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.523008:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.523011:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.523021:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.523028:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.523049:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760787.523787:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 73646080<>73650176 00000100:00100000:2.0:1789760787.523954:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565532288:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.540074:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985288 gen 3 00000008:00010000:2.0:1789760787.540128:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760787.540137:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.540149:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.540171:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760787.540200:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565532288:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.540385:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17980, 17980] 00000080:00200000:3.0:1789760787.540404:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.540411:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.540426:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73650176 i_blocks 143872 00000020:00200000:3.0:1789760787.540430:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.540451:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.540488:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73650176, count: 4096 00000080:00200000:3.0:1789760787.540495:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73650176, bytes: 4096 00000080:00200000:3.0:1789760787.540506:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.540515:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.540517:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73650176, 73654272) 0 0 00000080:00200000:3.0:1789760787.540521:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.540548:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17981, 17981] 00020000:00200000:3.0:1789760787.540558:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73650176 chunk: [73650176, 73654272) 73654272, 4096 00020000:00200000:3.0:1789760787.540562:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.540584:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17981, 17981] 00000008:00010000:3.0:1789760787.540603:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.540609:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.540621:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73650176, 73654272) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.540635:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73650176=0x463d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.540734:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.540737:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73654272 00000020:00010000:3.0:1789760787.540746:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.540750:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.540753:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.540756:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.540882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.540893:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.540899:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.540918:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760787.541041:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 73650176<>73654272 00000100:00100000:2.0:1789760787.541176:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760787.541194:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565532416:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.552330:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985289 gen 3 00000008:00010000:2.0:1789760787.552392:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760787.552401:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.552411:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.552434:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760787.552473:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565532416:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.552760:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17981, 17981] 00000080:00200000:3.0:1789760787.552775:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.552781:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.552805:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73654272 i_blocks 143880 00000020:00200000:3.0:1789760787.552809:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.552820:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.552856:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73654272, count: 4096 00000080:00200000:3.0:1789760787.552864:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73654272, bytes: 4096 00000080:00200000:3.0:1789760787.552873:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.552881:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.552884:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73654272, 73658368) 0 0 00000080:00200000:3.0:1789760787.552888:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.552905:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17982, 17982] 00020000:00200000:3.0:1789760787.552913:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73654272 chunk: [73654272, 73658368) 73658368, 4096 00020000:00200000:3.0:1789760787.552917:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.552937:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17982, 17982] 00000008:00010000:3.0:1789760787.552954:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.552960:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.552972:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73654272, 73658368) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.552986:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73654272=0x463e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.553045:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.553048:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73658368 00000020:00010000:3.0:1789760787.553056:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.553060:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.553063:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.553067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.553070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.553078:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.553085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.553102:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760787.553221:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 73654272<>73658368 00000100:00100000:2.0:1789760787.553396:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760787.553416:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565532544:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.564327:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985290 gen 3 00000008:00010000:2.0:1789760787.564376:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760787.564454:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.564467:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.564495:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760787.564525:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565532544:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.564811:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17982, 17982] 00000080:00200000:3.0:1789760787.564829:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.564835:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.564852:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73658368 i_blocks 143888 00000020:00200000:3.0:1789760787.564856:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.564865:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.564902:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73658368, count: 4096 00000080:00200000:3.0:1789760787.564909:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73658368, bytes: 4096 00000080:00200000:3.0:1789760787.564920:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.564929:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.564931:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73658368, 73662464) 0 0 00000080:00200000:3.0:1789760787.564936:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.564952:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17983, 17983] 00020000:00200000:3.0:1789760787.564960:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73658368 chunk: [73658368, 73662464) 73662464, 4096 00020000:00200000:3.0:1789760787.564965:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.564985:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17983, 17983] 00000008:00010000:3.0:1789760787.565003:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.565009:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.565020:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73658368, 73662464) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.565034:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73658368=0x463f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.565090:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.565093:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73662464 00000020:00010000:3.0:1789760787.565103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.565106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.565109:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.565113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.565116:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.565125:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.565131:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.565153:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760787.565287:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 73658368<>73662464 00000100:00100000:2.0:1789760787.565628:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760787.565674:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565532672:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.577005:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985291 gen 3 00000008:00010000:2.0:1789760787.577058:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760787.577067:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.577079:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.577210:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760787.577243:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565532672:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.578459:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17983, 17983] 00000080:00200000:3.0:1789760787.578480:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.578753:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.578954:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73662464 i_blocks 143896 00000020:00200000:2.0:1789760787.578960:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.578975:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.579012:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73662464, count: 4096 00000080:00200000:2.0:1789760787.579021:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73662464, bytes: 4096 00000080:00200000:2.0:1789760787.579036:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.579046:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.579049:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73662464, 73666560) 0 0 00000080:00200000:2.0:1789760787.579054:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.579073:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17984, 17984] 00020000:00200000:2.0:1789760787.579083:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73662464 chunk: [73662464, 73666560) 73666560, 4096 00020000:00200000:2.0:1789760787.579088:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.579109:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17984, 17984] 00000008:00010000:2.0:1789760787.579127:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.579134:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.579146:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73662464, 73666560) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.579160:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73662464=0x4640000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.579223:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.579226:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73666560 00000020:00010000:2.0:1789760787.579451:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.579457:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.579461:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.579465:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.579468:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.579478:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.579485:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.579885:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.580025:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 73662464<>73666560 00000100:00100000:3.0:1789760787.580101:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565532800:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.591911:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985292 gen 3 00000008:00010000:2.0:1789760787.591969:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760787.591978:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.591989:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.592313:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760787.592346:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565532800:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760787.592627:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17984, 17984] 00000080:00200000:0.0:1789760787.594414:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760787.594429:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760787.594459:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73666560 i_blocks 143904 00000020:00200000:0.0:1789760787.594465:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760787.594482:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760787.594519:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73666560, count: 4096 00000080:00200000:0.0:1789760787.594723:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73666560, bytes: 4096 00000080:00200000:0.0:1789760787.594742:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760787.594752:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760787.594754:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73666560, 73670656) 0 0 00000080:00200000:0.0:1789760787.594759:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760787.594780:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17985, 17985] 00020000:00200000:0.0:1789760787.594791:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73666560 chunk: [73666560, 73670656) 73670656, 4096 00020000:00200000:0.0:1789760787.594796:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760787.594820:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17985, 17985] 00000008:00010000:0.0:1789760787.594841:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760787.594847:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760787.594860:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73666560, 73670656) append 0 append lockpos 0 00000080:00200000:0.0:1789760787.594875:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73666560=0x4641000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760787.594948:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760787.594952:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73670656 00000020:00010000:0.0:1789760787.594960:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760787.594963:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760787.594966:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760787.594969:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760787.594971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760787.594980:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760787.594987:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760787.595018:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760787.595158:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73666560<>73670656 00000100:00100000:0.0:1789760787.595212:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760787.595307:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565532928:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.606725:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985293 gen 3 00000008:00010000:0.0:1789760787.606776:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760787.606785:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.606796:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.606808:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760787.606838:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565532928:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.607127:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17985, 17985] 00000080:00200000:1.0:1789760787.607144:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.607149:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.607166:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73670656 i_blocks 143912 00000020:00200000:1.0:1789760787.607170:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.607183:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.607221:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73670656, count: 4096 00000080:00200000:1.0:1789760787.607228:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73670656, bytes: 4096 00000080:00200000:1.0:1789760787.607253:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.607264:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.607266:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73670656, 73674752) 0 0 00000080:00200000:1.0:1789760787.607271:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.607291:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17986, 17986] 00020000:00200000:1.0:1789760787.607301:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73670656 chunk: [73670656, 73674752) 73674752, 4096 00020000:00200000:1.0:1789760787.607306:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.607330:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17986, 17986] 00000008:00010000:1.0:1789760787.607352:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.607359:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.607370:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73670656, 73674752) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.607384:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73670656=0x4642000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.607575:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.607578:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73674752 00000020:00010000:1.0:1789760787.607586:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.607590:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.607593:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.607596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.607598:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.607608:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.607616:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.607637:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760787.607823:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 73670656<>73674752 00000100:00100000:1.0:1789760787.607865:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760787.607887:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565533056:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.622984:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985294 gen 3 00000008:00010000:0.0:1789760787.623039:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760787.623047:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.623058:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.623080:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760787.623110:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565533056:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.623231:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17986, 17986] 00000080:00200000:1.0:1789760787.623246:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.623253:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.623267:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73674752 i_blocks 143920 00000020:00200000:1.0:1789760787.623270:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.623279:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.623315:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73674752, count: 4096 00000080:00200000:1.0:1789760787.623322:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73674752, bytes: 4096 00000080:00200000:1.0:1789760787.623334:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.623343:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.623345:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73674752, 73678848) 0 0 00000080:00200000:1.0:1789760787.623350:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.623365:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17987, 17987] 00020000:00200000:1.0:1789760787.623375:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73674752 chunk: [73674752, 73678848) 73678848, 4096 00020000:00200000:1.0:1789760787.623565:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.623587:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17987, 17987] 00000008:00010000:1.0:1789760787.623606:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.623613:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.623625:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73674752, 73678848) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.623805:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73674752=0x4643000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.623891:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.623894:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73678848 00000020:00010000:1.0:1789760787.623904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.623908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.623911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.623914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.623917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.623925:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.623933:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.623945:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760787.624076:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 73674752<>73678848 00000100:00100000:0.0:1789760787.624160:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565533184:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.636281:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985295 gen 3 00000008:00010000:0.0:1789760787.636333:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760787.636343:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.636353:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.636385:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760787.636415:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565533184:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.636579:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17987, 17987] 00000080:00200000:1.0:1789760787.636595:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.636601:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.636616:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73678848 i_blocks 143928 00000020:00200000:1.0:1789760787.636620:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.636629:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.636703:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73678848, count: 4096 00000080:00200000:1.0:1789760787.636711:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73678848, bytes: 4096 00000080:00200000:1.0:1789760787.636719:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.636727:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.636730:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73678848, 73682944) 0 0 00000080:00200000:1.0:1789760787.636734:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.636760:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17988, 17988] 00020000:00200000:1.0:1789760787.636769:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73678848 chunk: [73678848, 73682944) 73682944, 4096 00020000:00200000:1.0:1789760787.636775:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.636793:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17988, 17988] 00000008:00010000:1.0:1789760787.636811:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.636816:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.636828:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73678848, 73682944) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.636842:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73678848=0x4644000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.636899:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.636902:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73682944 00000020:00010000:1.0:1789760787.636910:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.636913:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.636916:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.636919:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.636922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.636929:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.636935:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.636952:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760787.637066:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 73678848<>73682944 00000100:00100000:0.0:1789760787.637201:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565533312:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.665993:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985296 gen 3 00000008:00010000:0.0:1789760787.666058:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760787.666067:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.666079:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.666106:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760787.666137:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565533312:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.667706:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17988, 17988] 00000080:00200000:1.0:1789760787.667731:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.667738:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.667755:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73682944 i_blocks 143936 00000020:00200000:1.0:1789760787.667760:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.667774:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.667809:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73682944, count: 4096 00000080:00200000:1.0:1789760787.667817:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73682944, bytes: 4096 00000080:00200000:1.0:1789760787.667833:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.667842:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.667845:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73682944, 73687040) 0 0 00000080:00200000:1.0:1789760787.667849:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.667867:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17989, 17989] 00020000:00200000:1.0:1789760787.667876:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73682944 chunk: [73682944, 73687040) 73687040, 4096 00020000:00200000:1.0:1789760787.667881:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.667907:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17989, 17989] 00000008:00010000:1.0:1789760787.667926:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.667932:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.667943:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73682944, 73687040) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.667958:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73682944=0x4645000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.668022:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.668025:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73687040 00000020:00010000:1.0:1789760787.668033:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.668036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.668039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.668042:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.668045:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.668053:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.668059:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.668081:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760787.668207:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 73682944<>73687040 00000100:00100000:0.0:1789760787.668347:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760787.668372:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565533440:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.684157:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985297 gen 3 00000008:00010000:0.0:1789760787.684208:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760787.684216:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.684228:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.684250:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760787.684281:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565533440:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.684545:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17989, 17989] 00000080:00200000:1.0:1789760787.684562:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.684568:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.684583:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73687040 i_blocks 143944 00000020:00200000:1.0:1789760787.684587:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.684597:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.684632:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73687040, count: 4096 00000080:00200000:1.0:1789760787.684639:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73687040, bytes: 4096 00000080:00200000:1.0:1789760787.684676:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.684685:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.684687:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73687040, 73691136) 0 0 00000080:00200000:1.0:1789760787.684691:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.684708:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17990, 17990] 00020000:00200000:1.0:1789760787.684717:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73687040 chunk: [73687040, 73691136) 73691136, 4096 00020000:00200000:1.0:1789760787.684722:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.684742:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17990, 17990] 00000008:00010000:1.0:1789760787.684759:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.684766:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.684777:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73687040, 73691136) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.684791:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73687040=0x4646000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.684847:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.684850:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73691136 00000020:00010000:1.0:1789760787.684857:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.684861:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.684865:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.684868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.684871:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.684878:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.684885:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.684900:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760787.685473:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 73687040<>73691136 00000100:00100000:0.0:1789760787.685568:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565533568:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.694941:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985298 gen 3 00000008:00010000:0.0:1789760787.694994:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760787.695002:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.695027:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.695046:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760787.695077:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565533568:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.695247:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17990, 17990] 00000080:00200000:1.0:1789760787.695264:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.695270:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.695285:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73691136 i_blocks 143952 00000020:00200000:1.0:1789760787.695288:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.695299:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.695334:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73691136, count: 4096 00000080:00200000:1.0:1789760787.695342:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73691136, bytes: 4096 00000080:00200000:1.0:1789760787.695353:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.695362:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.695364:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73691136, 73695232) 0 0 00000080:00200000:1.0:1789760787.695369:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.695387:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17991, 17991] 00020000:00200000:1.0:1789760787.695396:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73691136 chunk: [73691136, 73695232) 73695232, 4096 00020000:00200000:1.0:1789760787.695402:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.695423:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17991, 17991] 00000008:00010000:1.0:1789760787.695449:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.695455:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.695466:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73691136, 73695232) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.695481:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73691136=0x4647000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.695540:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.695543:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73695232 00000020:00010000:1.0:1789760787.695552:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.695555:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.695558:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.695562:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.695565:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.695572:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.695580:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.695596:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760787.695778:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 73691136<>73695232 00000100:00100000:0.0:1789760787.696166:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565533696:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.725002:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985299 gen 3 00000008:00010000:0.0:1789760787.725067:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760787.725076:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.725087:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.725107:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760787.725142:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565533696:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.725594:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17991, 17991] 00000080:00200000:1.0:1789760787.725610:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.725616:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.725630:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73695232 i_blocks 143960 00000020:00200000:1.0:1789760787.725634:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.725683:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.725718:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73695232, count: 4096 00000080:00200000:1.0:1789760787.725726:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73695232, bytes: 4096 00000080:00200000:1.0:1789760787.725737:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.725745:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.725748:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73695232, 73699328) 0 0 00000080:00200000:1.0:1789760787.725752:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.725769:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17992, 17992] 00020000:00200000:1.0:1789760787.725779:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73695232 chunk: [73695232, 73699328) 73699328, 4096 00020000:00200000:1.0:1789760787.725784:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.725804:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17992, 17992] 00000008:00010000:1.0:1789760787.725823:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.725830:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.725841:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73695232, 73699328) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.725854:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73695232=0x4648000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.725913:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.725916:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73699328 00000020:00010000:1.0:1789760787.725925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.725929:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.725932:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.725936:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.725940:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.725947:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.725954:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.725972:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760787.727242:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 73695232<>73699328 00000100:00100000:0.0:1789760787.727608:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760787.727636:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565533824:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.742749:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985300 gen 3 00000008:00010000:0.0:1789760787.742802:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760787.742809:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.742822:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.742853:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760787.742883:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565533824:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.743111:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17992, 17992] 00000080:00200000:2.0:1789760787.743127:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.743133:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.743148:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73699328 i_blocks 143968 00000020:00200000:2.0:1789760787.743152:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.743162:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.743207:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73699328, count: 4096 00000080:00200000:2.0:1789760787.743215:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73699328, bytes: 4096 00000080:00200000:2.0:1789760787.743228:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.743237:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.743239:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73699328, 73703424) 0 0 00000080:00200000:2.0:1789760787.743244:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.743259:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17993, 17993] 00020000:00200000:2.0:1789760787.743267:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73699328 chunk: [73699328, 73703424) 73703424, 4096 00020000:00200000:2.0:1789760787.743272:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.743291:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17993, 17993] 00000008:00010000:2.0:1789760787.743308:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.743315:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.743326:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73699328, 73703424) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.743340:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73699328=0x4649000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.743398:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.743402:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73703424 00000020:00010000:2.0:1789760787.743411:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.743414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.743417:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.743421:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.743424:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.743432:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.743448:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.743464:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.743589:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 73699328<>73703424 00000100:00100000:2.0:1789760787.743640:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760787.743710:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565533952:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.756049:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985301 gen 3 00000008:00010000:2.0:1789760787.756105:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760787.756115:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.756126:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.756155:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760787.756186:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565533952:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760787.756358:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17993, 17993] 00000080:00200000:0.0:1789760787.756375:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760787.756381:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760787.756406:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73703424 i_blocks 143976 00000020:00200000:0.0:1789760787.756410:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760787.756420:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760787.756466:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73703424, count: 4096 00000080:00200000:0.0:1789760787.756474:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73703424, bytes: 4096 00000080:00200000:0.0:1789760787.756496:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760787.756506:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760787.756509:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73703424, 73707520) 0 0 00000080:00200000:0.0:1789760787.756514:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760787.756531:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17994, 17994] 00020000:00200000:0.0:1789760787.756540:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73703424 chunk: [73703424, 73707520) 73707520, 4096 00020000:00200000:0.0:1789760787.756545:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760787.756566:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17994, 17994] 00000008:00010000:0.0:1789760787.756586:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760787.756592:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760787.756604:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73703424, 73707520) append 0 append lockpos 0 00000080:00200000:0.0:1789760787.756620:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73703424=0x464a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760787.756735:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760787.756738:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73707520 00000020:00010000:0.0:1789760787.756746:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760787.756750:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760787.756753:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760787.756757:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760787.756760:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760787.756768:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760787.756774:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760787.756793:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760787.757174:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73703424<>73707520 00000100:00100000:1.0:1789760787.757237:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760787.757771:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565534080:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.766811:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985302 gen 3 00000008:00010000:0.0:1789760787.766940:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760787.766950:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.766963:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.767032:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760787.767064:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565534080:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.772362:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17994, 17994] 00000080:00200000:1.0:1789760787.772492:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.772501:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.772591:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73707520 i_blocks 143984 00000020:00200000:1.0:1789760787.772598:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.772620:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.772671:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73707520, count: 4096 00000080:00200000:1.0:1789760787.772680:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73707520, bytes: 4096 00000080:00200000:1.0:1789760787.772697:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.772706:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.772709:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73707520, 73711616) 0 0 00000080:00200000:1.0:1789760787.772714:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.772733:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17995, 17995] 00020000:00200000:1.0:1789760787.772742:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73707520 chunk: [73707520, 73711616) 73711616, 4096 00020000:00200000:1.0:1789760787.772747:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.772770:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17995, 17995] 00000008:00010000:1.0:1789760787.772800:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.772807:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.772819:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73707520, 73711616) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.772833:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73707520=0x464b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.772907:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.772910:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73711616 00000020:00010000:1.0:1789760787.772918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.772921:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.772924:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.772927:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.772930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.772937:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.772944:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.772977:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760787.773293:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 73707520<>73711616 00000100:00100000:0.0:1789760787.773373:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565534208:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.788941:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985303 gen 3 00000008:00010000:0.0:1789760787.788990:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760787.788998:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.789020:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.789041:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760787.789071:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565534208:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.789248:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17995, 17995] 00000080:00200000:1.0:1789760787.789264:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.789271:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.789285:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73711616 i_blocks 143992 00000020:00200000:1.0:1789760787.789289:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.789299:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.789333:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73711616, count: 4096 00000080:00200000:1.0:1789760787.789340:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73711616, bytes: 4096 00000080:00200000:1.0:1789760787.789351:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.789359:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.789362:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73711616, 73715712) 0 0 00000080:00200000:1.0:1789760787.789367:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.789393:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17996, 17996] 00020000:00200000:1.0:1789760787.789403:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73711616 chunk: [73711616, 73715712) 73715712, 4096 00020000:00200000:1.0:1789760787.789408:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.789428:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17996, 17996] 00000008:00010000:1.0:1789760787.789455:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.789461:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.789473:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73711616, 73715712) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.789597:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73711616=0x464c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.789701:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.789705:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73715712 00000020:00010000:1.0:1789760787.789713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.789717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.789720:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.789723:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.789726:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.789734:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.789741:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.789761:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760787.789904:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 73711616<>73715712 00000100:00100000:0.0:1789760787.790131:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565534336:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.802429:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985304 gen 3 00000008:00010000:0.0:1789760787.802500:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760787.802509:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.802521:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.802539:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760787.802568:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565534336:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.802684:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17996, 17996] 00000080:00200000:1.0:1789760787.802700:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.802707:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.802721:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73715712 i_blocks 144000 00000020:00200000:1.0:1789760787.802726:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.802735:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.802766:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73715712, count: 4096 00000080:00200000:1.0:1789760787.802773:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73715712, bytes: 4096 00000080:00200000:1.0:1789760787.802783:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.802802:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.802805:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73715712, 73719808) 0 0 00000080:00200000:1.0:1789760787.802810:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.802826:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17997, 17997] 00020000:00200000:1.0:1789760787.802835:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73715712 chunk: [73715712, 73719808) 73719808, 4096 00020000:00200000:1.0:1789760787.802839:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.802857:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17997, 17997] 00000008:00010000:1.0:1789760787.802873:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.802879:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.802898:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73715712, 73719808) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.802914:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73715712=0x464d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.802973:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.802977:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73719808 00000020:00010000:1.0:1789760787.802984:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.802987:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.802990:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.802993:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.802996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.803004:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.803132:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.803150:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760787.803263:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 73715712<>73719808 00000100:00100000:0.0:1789760787.803359:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760787.803376:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565534464:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.828279:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985305 gen 3 00000008:00010000:0.0:1789760787.828326:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760787.828335:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.828345:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.828367:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760787.828407:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565534464:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.828529:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17997, 17997] 00000080:00200000:1.0:1789760787.828545:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.828550:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.828567:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73719808 i_blocks 144008 00000020:00200000:1.0:1789760787.828571:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.828581:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.828614:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73719808, count: 4096 00000080:00200000:1.0:1789760787.828622:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73719808, bytes: 4096 00000080:00200000:1.0:1789760787.828632:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.828641:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.828683:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73719808, 73723904) 0 0 00000080:00200000:1.0:1789760787.828689:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.828707:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17998, 17998] 00020000:00200000:1.0:1789760787.828716:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73719808 chunk: [73719808, 73723904) 73723904, 4096 00020000:00200000:1.0:1789760787.828720:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.828742:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17998, 17998] 00000008:00010000:1.0:1789760787.828761:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.828767:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.828779:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73719808, 73723904) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.828805:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73719808=0x464e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.828862:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.828865:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73723904 00000020:00010000:1.0:1789760787.828873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.828876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.828880:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.828883:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.828886:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.828893:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.828900:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.828917:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760787.829038:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 73719808<>73723904 00000100:00100000:0.0:1789760787.829358:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565534592:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.846109:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985306 gen 3 00000008:00010000:0.0:1789760787.846161:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760787.846170:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.846182:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.846203:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760787.846233:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565534592:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.846345:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17998, 17998] 00000080:00200000:1.0:1789760787.846363:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.846369:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.846385:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73723904 i_blocks 144016 00000020:00200000:1.0:1789760787.846389:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.846398:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.846433:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73723904, count: 4096 00000080:00200000:1.0:1789760787.846450:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73723904, bytes: 4096 00000080:00200000:1.0:1789760787.846461:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.846469:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.846472:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73723904, 73728000) 0 0 00000080:00200000:1.0:1789760787.846476:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.846573:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [17999, 17999] 00020000:00200000:1.0:1789760787.846582:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73723904 chunk: [73723904, 73728000) 73728000, 4096 00020000:00200000:1.0:1789760787.846587:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.846608:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [17999, 17999] 00000008:00010000:1.0:1789760787.846625:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.846632:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.846688:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73723904, 73728000) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.846703:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73723904=0x464f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.846762:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.846765:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73728000 00000020:00010000:1.0:1789760787.846772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.846776:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.846779:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.846859:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.846863:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.846873:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.846880:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.846898:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760787.847023:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 73723904<>73728000 00000100:00100000:0.0:1789760787.847120:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760787.847139:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565534720:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.859680:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985307 gen 3 00000008:00010000:0.0:1789760787.859735:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760787.859745:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.859757:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.859769:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760787.859798:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565534720:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.859904:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [17999, 17999] 00000080:00200000:1.0:1789760787.859920:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.859927:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.859943:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73728000 i_blocks 144024 00000020:00200000:1.0:1789760787.859947:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.859956:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.859985:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73728000, count: 4096 00000080:00200000:1.0:1789760787.859993:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73728000, bytes: 4096 00000080:00200000:1.0:1789760787.860002:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.860011:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.860025:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73728000, 73732096) 0 0 00000080:00200000:1.0:1789760787.860030:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.860047:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18000, 18000] 00020000:00200000:1.0:1789760787.860055:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73728000 chunk: [73728000, 73732096) 73732096, 4096 00020000:00200000:1.0:1789760787.860060:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.860081:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18000, 18000] 00000008:00010000:1.0:1789760787.860099:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.860106:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.860119:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73728000, 73732096) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.860132:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73728000=0x4650000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.860191:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.860195:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73732096 00000020:00010000:1.0:1789760787.860203:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.860207:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.860210:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.860213:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.860217:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.860225:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.860231:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.860248:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760787.860371:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 73728000<>73732096 00000100:00100000:0.0:1789760787.860533:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565534848:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.870754:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985308 gen 3 00000008:00010000:0.0:1789760787.870814:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760787.870823:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.870834:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.870854:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760787.870886:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565534848:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.871105:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18000, 18000] 00000080:00200000:1.0:1789760787.871122:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.871128:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.871143:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73732096 i_blocks 144032 00000020:00200000:1.0:1789760787.871147:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.871157:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.871192:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73732096, count: 4096 00000080:00200000:1.0:1789760787.871199:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73732096, bytes: 4096 00000080:00200000:1.0:1789760787.871209:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.871216:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.871219:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73732096, 73736192) 0 0 00000080:00200000:1.0:1789760787.871224:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.871241:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18001, 18001] 00020000:00200000:1.0:1789760787.871249:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73732096 chunk: [73732096, 73736192) 73736192, 4096 00020000:00200000:1.0:1789760787.871254:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.871273:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18001, 18001] 00000008:00010000:1.0:1789760787.871291:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.871297:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.871309:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73732096, 73736192) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.871323:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73732096=0x4651000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.871395:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.871398:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73736192 00000020:00010000:1.0:1789760787.871407:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.871410:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.871414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.871418:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.871421:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.871428:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.871435:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.871458:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760787.871594:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 73732096<>73736192 00000100:00100000:0.0:1789760787.872043:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565534976:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.887711:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985309 gen 3 00000008:00010000:0.0:1789760787.887761:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760787.887770:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.887781:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.887805:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760787.887834:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565534976:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.888128:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18001, 18001] 00000080:00200000:1.0:1789760787.888146:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.888152:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.888166:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73736192 i_blocks 144040 00000020:00200000:1.0:1789760787.888170:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.888180:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.888214:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73736192, count: 4096 00000080:00200000:1.0:1789760787.888221:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73736192, bytes: 4096 00000080:00200000:1.0:1789760787.888234:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.888243:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.888246:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73736192, 73740288) 0 0 00000080:00200000:1.0:1789760787.888251:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.888267:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18002, 18002] 00020000:00200000:1.0:1789760787.888277:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73736192 chunk: [73736192, 73740288) 73740288, 4096 00020000:00200000:1.0:1789760787.888281:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.888301:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18002, 18002] 00000008:00010000:1.0:1789760787.888319:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.888326:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.888338:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73736192, 73740288) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.888351:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73736192=0x4652000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.888421:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.888424:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73740288 00000020:00010000:1.0:1789760787.888433:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.888445:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.888448:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.888451:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.888454:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.888462:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.888469:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.888547:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760787.888725:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 73736192<>73740288 00000100:00100000:0.0:1789760787.889048:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565535104:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.906201:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985310 gen 3 00000008:00010000:0.0:1789760787.906256:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760787.906265:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.906277:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.906305:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760787.906336:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565535104:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.906541:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18002, 18002] 00000080:00200000:1.0:1789760787.906560:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.906566:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.906580:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73740288 i_blocks 144048 00000020:00200000:1.0:1789760787.906584:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.906593:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.906628:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73740288, count: 4096 00000080:00200000:1.0:1789760787.906635:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73740288, bytes: 4096 00000080:00200000:1.0:1789760787.906690:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.906700:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.906703:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73740288, 73744384) 0 0 00000080:00200000:1.0:1789760787.906708:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.906725:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18003, 18003] 00020000:00200000:1.0:1789760787.906735:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73740288 chunk: [73740288, 73744384) 73744384, 4096 00020000:00200000:1.0:1789760787.906739:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.906760:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18003, 18003] 00000008:00010000:1.0:1789760787.906777:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.906784:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.906795:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73740288, 73744384) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.906809:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73740288=0x4653000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.906866:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.906869:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73744384 00000020:00010000:1.0:1789760787.906878:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.906882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.906885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.906889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.906892:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.906902:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.906909:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.906927:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760787.907054:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 73740288<>73744384 00000100:00100000:0.0:1789760787.907301:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565535232:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.924361:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985311 gen 3 00000008:00010000:0.0:1789760787.924423:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760787.924431:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760787.924451:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760787.924471:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760787.924502:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565535232:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.924671:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18003, 18003] 00000080:00200000:1.0:1789760787.924689:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.924696:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.924713:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73744384 i_blocks 144056 00000020:00200000:1.0:1789760787.924717:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.924726:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.924762:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73744384, count: 4096 00000080:00200000:1.0:1789760787.924769:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73744384, bytes: 4096 00000080:00200000:1.0:1789760787.924779:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.924859:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.924862:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73744384, 73748480) 0 0 00000080:00200000:1.0:1789760787.924868:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.924885:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18004, 18004] 00020000:00200000:1.0:1789760787.924893:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73744384 chunk: [73744384, 73748480) 73748480, 4096 00020000:00200000:1.0:1789760787.924898:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.924918:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18004, 18004] 00000008:00010000:1.0:1789760787.924936:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.924943:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.924954:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73744384, 73748480) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.924968:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73744384=0x4654000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.925101:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.925105:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73748480 00000020:00010000:1.0:1789760787.925114:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.925118:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.925122:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.925125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.925128:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.925136:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.925144:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.925164:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760787.925289:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 73744384<>73748480 00000100:00100000:0.0:1789760787.925367:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565535360:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760787.946039:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985312 gen 3 00000008:00010000:1.0:1789760787.949014:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760787.949032:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760787.949046:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760787.949084:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760787.949135:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565535360:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760787.949154:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18004, 18004] 00000080:00200000:2.0:1789760787.949174:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760787.949181:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760787.949197:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73748480 i_blocks 144064 00000020:00200000:2.0:1789760787.949201:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760787.949212:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760787.949249:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73748480, count: 4096 00000080:00200000:2.0:1789760787.949257:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73748480, bytes: 4096 00000080:00200000:2.0:1789760787.949272:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760787.949280:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760787.949283:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73748480, 73752576) 0 0 00000080:00200000:2.0:1789760787.949287:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760787.949305:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18005, 18005] 00020000:00200000:2.0:1789760787.949314:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73748480 chunk: [73748480, 73752576) 73752576, 4096 00020000:00200000:2.0:1789760787.949319:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760787.949339:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18005, 18005] 00000008:00010000:2.0:1789760787.949358:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760787.949365:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760787.949376:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73748480, 73752576) append 0 append lockpos 0 00000080:00200000:2.0:1789760787.949403:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73748480=0x4655000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760787.949475:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.949479:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73752576 00000020:00010000:2.0:1789760787.949498:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.949502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.949506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.949509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.949513:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.949521:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.949528:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.949545:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.949716:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 73748480<>73752576 00000100:00100000:2.0:1789760787.949759:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760787.949779:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565535488:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.968281:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985313 gen 3 00000008:00010000:2.0:1789760787.968330:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760787.968338:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.968349:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.968367:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760787.968588:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565535488:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760787.968880:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18005, 18005] 00000080:00200000:3.0:1789760787.968896:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.968903:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.968919:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73752576 i_blocks 144072 00000020:00200000:3.0:1789760787.968922:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.968932:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.968996:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73752576, count: 4096 00000080:00200000:3.0:1789760787.969005:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73752576, bytes: 4096 00000080:00200000:3.0:1789760787.969086:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.969094:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.969097:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73752576, 73756672) 0 0 00000080:00200000:3.0:1789760787.969102:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.969119:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18006, 18006] 00020000:00200000:3.0:1789760787.969129:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73752576 chunk: [73752576, 73756672) 73756672, 4096 00020000:00200000:3.0:1789760787.969133:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.969156:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18006, 18006] 00000008:00010000:3.0:1789760787.969174:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.969181:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.969193:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73752576, 73756672) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.969206:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73752576=0x4656000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760787.969266:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760787.969269:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73756672 00000020:00010000:3.0:1789760787.969278:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760787.969281:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.969284:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760787.969288:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760787.969291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760787.969300:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760787.969307:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760787.969326:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760787.970750:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 73752576<>73756672 00000100:00100000:2.0:1789760787.970910:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565535616:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760787.985090:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985314 gen 3 00000008:00010000:2.0:1789760787.985574:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760787.985583:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.985595:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.985622:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000080:00200000:3.0:1789760787.985894:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18006, 18006] 00000080:00200000:3.0:1789760787.985984:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760787.985991:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760787.986007:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73756672 i_blocks 144080 00000020:00200000:3.0:1789760787.986011:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760787.986021:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760787.986055:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73756672, count: 4096 00000080:00200000:3.0:1789760787.986063:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73756672, bytes: 4096 00000080:00200000:3.0:1789760787.986074:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760787.986083:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760787.986085:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73756672, 73760768) 0 0 00000080:00200000:3.0:1789760787.986089:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760787.986112:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18007, 18007] 00020000:00200000:3.0:1789760787.986122:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73756672 chunk: [73756672, 73760768) 73760768, 4096 00020000:00200000:3.0:1789760787.986126:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760787.986158:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18007, 18007] 00000008:00010000:3.0:1789760787.986175:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760787.986182:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e8b0, osc lock ffff93f8457e6f70, flags 0x0 00000080:00200000:3.0:1789760787.986193:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73756672, 73760768) append 0 append lockpos 0 00000080:00200000:3.0:1789760787.986207:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73756672=0x4657000, pages 1 (max 131072), lockless, parallel 00000100:00100000:2.0:1789760787.986334:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565535616:192.168.203.147@tcp:4:dd.0 00000020:00200000:2.0:1789760787.987808:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760787.987817:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73760768 00000020:00010000:2.0:1789760787.987832:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760787.987836:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.987840:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e8b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760787.987843:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760787.987850:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e8b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760787.987864:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760787.987872:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760787.987890:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760787.988091:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 73756672<>73760768 00000100:00100000:3.0:1789760787.988223:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565535744:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760787.995070:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760787.997111:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985315 gen 3 00000008:00010000:2.0:1789760787.997155:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760787.997163:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760787.997175:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760787.997253:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760787.997283:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565535744:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760787.997640:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18007, 18007] 00000080:00200000:1.0:1789760787.997799:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760787.997805:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760787.997821:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73760768 i_blocks 144088 00000020:00200000:1.0:1789760787.997825:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760787.997835:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760787.997869:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73760768, count: 4096 00000080:00200000:1.0:1789760787.997877:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73760768, bytes: 4096 00000080:00200000:1.0:1789760787.997887:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760787.997895:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760787.997897:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73760768, 73764864) 0 0 00000080:00200000:1.0:1789760787.997913:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760787.997930:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18008, 18008] 00020000:00200000:1.0:1789760787.997939:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73760768 chunk: [73760768, 73764864) 73764864, 4096 00020000:00200000:1.0:1789760787.997944:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760787.997964:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18008, 18008] 00000008:00010000:1.0:1789760787.997988:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760787.997993:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760787.998004:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73760768, 73764864) append 0 append lockpos 0 00000080:00200000:1.0:1789760787.998028:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73760768=0x4658000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760787.998088:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760787.998091:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73764864 00000020:00010000:1.0:1789760787.998100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760787.998104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.998108:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760787.998111:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760787.998114:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760787.998121:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760787.998128:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760787.998147:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760787.999144:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 73760768<>73764864 00000100:00100000:1.0:1789760787.999197:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760787.999219:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565535872:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.013231:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985316 gen 3 00000008:00010000:0.0:1789760788.013282:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760788.013291:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.013301:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.013378:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760788.013407:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565535872:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.017028:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18008, 18008] 00000080:00200000:2.0:1789760788.017060:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.017066:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.017085:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73764864 i_blocks 144096 00000020:00200000:2.0:1789760788.017090:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.017110:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.017149:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73764864, count: 4096 00000080:00200000:2.0:1789760788.017159:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73764864, bytes: 4096 00000080:00200000:2.0:1789760788.017175:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.017183:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.017186:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73764864, 73768960) 0 0 00000080:00200000:2.0:1789760788.017190:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.017206:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18009, 18009] 00020000:00200000:2.0:1789760788.017215:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73764864 chunk: [73764864, 73768960) 73768960, 4096 00020000:00200000:2.0:1789760788.017220:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.017242:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18009, 18009] 00000008:00010000:2.0:1789760788.017262:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.017268:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.017281:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73764864, 73768960) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.017298:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73764864=0x4659000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.017365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.017368:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73768960 00000020:00010000:2.0:1789760788.017376:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.017380:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.017382:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.017386:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.017389:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.017397:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.017403:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.017430:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760788.020814:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 73764864<>73768960 00000100:00100000:2.0:1789760788.020979:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565536000:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.044019:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985317 gen 3 00000008:00010000:2.0:1789760788.044069:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760788.044079:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.044088:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.044113:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760788.044140:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565536000:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.044149:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18009, 18009] 00000080:00200000:3.0:1789760788.044168:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.044173:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.044190:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73768960 i_blocks 144104 00000020:00200000:3.0:1789760788.044194:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.044205:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.044249:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73768960, count: 4096 00000080:00200000:3.0:1789760788.044257:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73768960, bytes: 4096 00000080:00200000:3.0:1789760788.044267:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.044275:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.044277:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73768960, 73773056) 0 0 00000080:00200000:3.0:1789760788.044281:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.044297:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18010, 18010] 00020000:00200000:3.0:1789760788.044314:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73768960 chunk: [73768960, 73773056) 73773056, 4096 00020000:00200000:3.0:1789760788.044320:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.044339:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18010, 18010] 00000008:00010000:3.0:1789760788.044355:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.044361:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.044372:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73768960, 73773056) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.044385:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73768960=0x465a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.044451:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.044454:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73773056 00000020:00010000:3.0:1789760788.044463:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.044466:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.044470:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.044474:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.044477:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.044485:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.044492:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760788.044507:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760788.044634:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 73768960<>73773056 00000100:00100000:2.0:1789760788.044802:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565536128:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.058958:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985318 gen 3 00000008:00010000:2.0:1789760788.059010:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760788.059020:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.059031:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.059111:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760788.059142:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565536128:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.060056:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18010, 18010] 00000080:00200000:3.0:1789760788.060077:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.060083:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.060099:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73773056 i_blocks 144112 00000020:00200000:3.0:1789760788.060103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.060114:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.060148:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73773056, count: 4096 00000080:00200000:3.0:1789760788.060328:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73773056, bytes: 4096 00000080:00200000:3.0:1789760788.060341:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.060350:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.060352:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73773056, 73777152) 0 0 00000080:00200000:3.0:1789760788.060357:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.060429:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18011, 18011] 00020000:00200000:3.0:1789760788.060448:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73773056 chunk: [73773056, 73777152) 73777152, 4096 00020000:00200000:3.0:1789760788.060453:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.060579:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18011, 18011] 00000008:00010000:3.0:1789760788.060598:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.060604:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.060618:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73773056, 73777152) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.060634:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73773056=0x465b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.060719:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.060722:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73777152 00000020:00010000:3.0:1789760788.060731:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.060734:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.060737:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.060857:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.060861:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.060872:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.060879:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760788.060899:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760788.061026:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 73773056<>73777152 00000100:00100000:2.0:1789760788.061111:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565536256:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.072338:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985319 gen 3 00000008:00010000:2.0:1789760788.072398:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760788.072407:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.072419:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.072449:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760788.072478:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565536256:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.072584:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18011, 18011] 00000080:00200000:3.0:1789760788.072600:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.072606:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.072620:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73777152 i_blocks 144120 00000020:00200000:3.0:1789760788.072624:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.072633:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.072715:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73777152, count: 4096 00000080:00200000:3.0:1789760788.072724:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73777152, bytes: 4096 00000080:00200000:3.0:1789760788.072734:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.072743:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.072745:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73777152, 73781248) 0 0 00000080:00200000:3.0:1789760788.072750:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.072767:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18012, 18012] 00020000:00200000:3.0:1789760788.072776:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73777152 chunk: [73777152, 73781248) 73781248, 4096 00020000:00200000:3.0:1789760788.072780:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.072800:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18012, 18012] 00000008:00010000:3.0:1789760788.072818:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.072825:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.072836:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73777152, 73781248) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.072849:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73777152=0x465c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.072908:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.072911:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73781248 00000020:00010000:3.0:1789760788.072920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.072924:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.072927:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.072932:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.072934:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.072942:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.072949:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760788.072966:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760788.073095:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 73777152<>73781248 00000100:00100000:2.0:1789760788.073365:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565536384:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.083530:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985320 gen 3 00000008:00010000:2.0:1789760788.083584:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760788.083593:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.083604:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.083711:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760788.083741:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565536384:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.087053:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18012, 18012] 00000080:00200000:3.0:1789760788.087079:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.087086:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.087104:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73781248 i_blocks 144128 00000020:00200000:3.0:1789760788.087110:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.087127:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.087163:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73781248, count: 4096 00000080:00200000:3.0:1789760788.087172:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73781248, bytes: 4096 00000080:00200000:3.0:1789760788.087187:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.087198:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.087200:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73781248, 73785344) 0 0 00000080:00200000:3.0:1789760788.087205:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.087225:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18013, 18013] 00020000:00200000:3.0:1789760788.087234:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73781248 chunk: [73781248, 73785344) 73785344, 4096 00020000:00200000:3.0:1789760788.087239:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.087265:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18013, 18013] 00000008:00010000:3.0:1789760788.087288:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.087295:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.087311:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73781248, 73785344) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.087328:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73781248=0x465d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.089580:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.089593:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73785344 00000020:00010000:3.0:1789760788.089610:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.089615:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.089621:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.089627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.089629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.089641:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.089666:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760788.089693:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760788.090524:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 73781248<>73785344 00000100:00100000:2.0:1789760788.090636:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565536512:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.099322:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985321 gen 3 00000008:00010000:2.0:1789760788.099371:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760788.099449:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.099462:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.099492:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760788.099525:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565536512:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.099632:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18013, 18013] 00000080:00200000:3.0:1789760788.099692:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.099699:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.099716:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73785344 i_blocks 144136 00000020:00200000:3.0:1789760788.099719:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.099730:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.099764:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73785344, count: 4096 00000080:00200000:3.0:1789760788.099772:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73785344, bytes: 4096 00000080:00200000:3.0:1789760788.099782:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.099790:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.099792:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73785344, 73789440) 0 0 00000080:00200000:3.0:1789760788.099797:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.099812:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18014, 18014] 00020000:00200000:3.0:1789760788.099821:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73785344 chunk: [73785344, 73789440) 73789440, 4096 00020000:00200000:3.0:1789760788.099825:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.099844:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18014, 18014] 00000008:00010000:3.0:1789760788.099862:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.099868:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.099879:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73785344, 73789440) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.099891:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73785344=0x465e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.099947:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.099950:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73789440 00000020:00010000:3.0:1789760788.099958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.099961:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.099964:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.099968:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.099971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.099979:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.099987:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760788.100004:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760788.100124:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 73785344<>73789440 00000100:00100000:2.0:1789760788.100642:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565536640:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.118087:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985322 gen 3 00000008:00010000:2.0:1789760788.118139:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760788.118149:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.118160:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.118184:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:2.0:1789760788.118215:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565536640:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.118323:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18014, 18014] 00000080:00200000:3.0:1789760788.118340:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.118346:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.118360:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73789440 i_blocks 144144 00000020:00200000:3.0:1789760788.118364:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.118385:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.118417:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73789440, count: 4096 00000080:00200000:3.0:1789760788.118424:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73789440, bytes: 4096 00000080:00200000:3.0:1789760788.118434:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.118454:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.118457:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73789440, 73793536) 0 0 00000080:00200000:3.0:1789760788.118462:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.118479:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18015, 18015] 00020000:00200000:3.0:1789760788.118497:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73789440 chunk: [73789440, 73793536) 73793536, 4096 00020000:00200000:3.0:1789760788.118502:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.118523:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18015, 18015] 00000008:00010000:3.0:1789760788.118541:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.118547:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.118559:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73789440, 73793536) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.118574:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73789440=0x465f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.118632:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.118635:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73793536 00000020:00010000:3.0:1789760788.118692:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.118696:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.118699:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.118704:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.118707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.118715:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.118722:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760788.118739:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760788.119065:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 73789440<>73793536 00000100:00100000:3.0:1789760788.119102:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760788.119118:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565536768:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.134186:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985323 gen 3 00000008:00010000:3.0:1789760788.134243:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:3.0:1789760788.134254:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.134268:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.134383:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:3.0:1789760788.134416:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565536768:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.134691:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18015, 18015] 00000080:00200000:2.0:1789760788.134710:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.134716:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.134732:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73793536 i_blocks 144152 00000020:00200000:2.0:1789760788.134736:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.134882:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.134921:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73793536, count: 4096 00000080:00200000:2.0:1789760788.134930:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73793536, bytes: 4096 00000080:00200000:2.0:1789760788.134944:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.134952:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.134955:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73793536, 73797632) 0 0 00000080:00200000:2.0:1789760788.134960:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.134977:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18016, 18016] 00020000:00200000:2.0:1789760788.134988:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73793536 chunk: [73793536, 73797632) 73797632, 4096 00020000:00200000:2.0:1789760788.134993:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.135013:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18016, 18016] 00000008:00010000:2.0:1789760788.135032:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.135038:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.135051:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73793536, 73797632) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.135067:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73793536=0x4660000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.135127:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.135130:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73797632 00000020:00010000:2.0:1789760788.135138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.135142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.135145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.135149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.135152:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.135159:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.135166:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.135187:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760788.135322:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 73793536<>73797632 00000100:00100000:3.0:1789760788.135476:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565536896:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.144681:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985324 gen 3 00000008:00010000:3.0:1789760788.144736:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760788.144756:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.144768:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.144850:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760788.144881:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565536896:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.148337:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18016, 18016] 00000080:00200000:2.0:1789760788.148364:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.148429:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.148459:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73797632 i_blocks 144160 00000020:00200000:2.0:1789760788.148466:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.148483:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.148582:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73797632, count: 4096 00000080:00200000:2.0:1789760788.148591:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73797632, bytes: 4096 00000080:00200000:2.0:1789760788.148606:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.148615:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.148618:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73797632, 73801728) 0 0 00000080:00200000:2.0:1789760788.148623:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.148640:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18017, 18017] 00020000:00200000:2.0:1789760788.148667:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73797632 chunk: [73797632, 73801728) 73801728, 4096 00020000:00200000:2.0:1789760788.148673:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.148697:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18017, 18017] 00000008:00010000:2.0:1789760788.148718:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.148725:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.148739:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73797632, 73801728) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.148753:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73797632=0x4661000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.149073:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.149076:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73801728 00000020:00010000:2.0:1789760788.149086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.149090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.149093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.149096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.149100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.149109:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.149116:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.149137:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760788.149263:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 73797632<>73801728 00000100:00100000:3.0:1789760788.149353:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565537024:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.166582:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985325 gen 3 00000008:00010000:3.0:1789760788.166632:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760788.166640:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.166692:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.166715:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760788.166743:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565537024:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.166905:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18017, 18017] 00000080:00200000:2.0:1789760788.166921:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.166928:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.166944:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73801728 i_blocks 144168 00000020:00200000:2.0:1789760788.166948:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.166957:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.166992:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73801728, count: 4096 00000080:00200000:2.0:1789760788.167000:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73801728, bytes: 4096 00000080:00200000:2.0:1789760788.167010:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.167019:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.167022:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73801728, 73805824) 0 0 00000080:00200000:2.0:1789760788.167026:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.167041:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18018, 18018] 00020000:00200000:2.0:1789760788.167049:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73801728 chunk: [73801728, 73805824) 73805824, 4096 00020000:00200000:2.0:1789760788.167053:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.167073:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18018, 18018] 00000008:00010000:2.0:1789760788.167090:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.167096:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.167108:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73801728, 73805824) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.167121:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73801728=0x4662000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.167174:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.167177:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73805824 00000020:00010000:2.0:1789760788.167185:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.167188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.167191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.167195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.167198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.167205:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.167211:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.167228:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760788.167347:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 73801728<>73805824 00000100:00100000:3.0:1789760788.167473:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565537152:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.177116:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985326 gen 3 00000008:00010000:3.0:1789760788.177170:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760788.177178:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.177190:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.177273:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760788.177304:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565537152:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.177451:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18018, 18018] 00000080:00200000:2.0:1789760788.177469:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.177579:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.177597:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73805824 i_blocks 144176 00000020:00200000:2.0:1789760788.177601:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.177611:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.177673:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73805824, count: 4096 00000080:00200000:2.0:1789760788.177681:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73805824, bytes: 4096 00000080:00200000:2.0:1789760788.177690:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.177699:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.177701:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73805824, 73809920) 0 0 00000080:00200000:2.0:1789760788.177705:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.177722:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18019, 18019] 00020000:00200000:2.0:1789760788.177730:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73805824 chunk: [73805824, 73809920) 73809920, 4096 00020000:00200000:2.0:1789760788.177735:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.177755:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18019, 18019] 00000008:00010000:2.0:1789760788.177773:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.177853:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.177868:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73805824, 73809920) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.177883:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73805824=0x4663000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.177939:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.177942:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73809920 00000020:00010000:2.0:1789760788.177950:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.177954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.177956:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.177960:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.177962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.177971:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.177979:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.177999:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760788.178123:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 73805824<>73809920 00000100:00100000:3.0:1789760788.178203:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565537280:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.185204:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985327 gen 3 00000008:00010000:2.0:1789760788.185253:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760788.185262:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.185273:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.185298:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:2.0:1789760788.185328:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565537280:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760788.185450:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18019, 18019] 00000080:00200000:0.0:1789760788.185467:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760788.185473:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760788.185499:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73809920 i_blocks 144184 00000020:00200000:0.0:1789760788.185503:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760788.185514:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760788.185551:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73809920, count: 4096 00000080:00200000:0.0:1789760788.185558:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73809920, bytes: 4096 00000080:00200000:0.0:1789760788.185570:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760788.185579:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760788.185581:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73809920, 73814016) 0 0 00000080:00200000:0.0:1789760788.185587:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760788.185605:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18020, 18020] 00020000:00200000:0.0:1789760788.185614:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73809920 chunk: [73809920, 73814016) 73814016, 4096 00020000:00200000:0.0:1789760788.185619:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760788.185641:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18020, 18020] 00000008:00010000:0.0:1789760788.185683:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760788.185690:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760788.185704:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73809920, 73814016) append 0 append lockpos 0 00000080:00200000:0.0:1789760788.185717:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73809920=0x4664000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760788.185776:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760788.185779:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73814016 00000020:00010000:0.0:1789760788.185787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760788.185791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760788.185794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760788.185797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760788.185800:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760788.185807:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760788.185814:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760788.185829:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760788.185944:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73809920<>73814016 00000100:00100000:1.0:1789760788.186017:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565537408:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.195975:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985328 gen 3 00000008:00010000:0.0:1789760788.196025:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760788.196033:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.196045:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.196071:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760788.196100:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565537408:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.196214:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18020, 18020] 00000080:00200000:2.0:1789760788.196230:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.196236:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.196250:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73814016 i_blocks 144192 00000020:00200000:2.0:1789760788.196254:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.196266:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.196301:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73814016, count: 4096 00000080:00200000:2.0:1789760788.196308:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73814016, bytes: 4096 00000080:00200000:2.0:1789760788.196319:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.196327:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.196329:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73814016, 73818112) 0 0 00000080:00200000:2.0:1789760788.196333:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.196349:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18021, 18021] 00020000:00200000:2.0:1789760788.196358:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73814016 chunk: [73814016, 73818112) 73818112, 4096 00020000:00200000:2.0:1789760788.196362:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.196452:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18021, 18021] 00000008:00010000:2.0:1789760788.196472:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.196478:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.196840:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73814016, 73818112) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.196856:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73814016=0x4665000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.196917:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.196920:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73818112 00000020:00010000:2.0:1789760788.196930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.196934:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.196937:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.196941:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.196943:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.196952:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.196959:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.196978:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760788.197102:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 73814016<>73818112 00000100:00100000:3.0:1789760788.197182:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565537536:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.204572:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985329 gen 3 00000008:00010000:3.0:1789760788.204625:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760788.204634:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.204661:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.204684:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760788.204712:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565537536:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.204897:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18021, 18021] 00000080:00200000:2.0:1789760788.204914:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.204920:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.204936:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73818112 i_blocks 144200 00000020:00200000:2.0:1789760788.204940:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.204950:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.204985:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73818112, count: 4096 00000080:00200000:2.0:1789760788.204993:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73818112, bytes: 4096 00000080:00200000:2.0:1789760788.205004:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.205012:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.205015:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73818112, 73822208) 0 0 00000080:00200000:2.0:1789760788.205020:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.205037:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18022, 18022] 00020000:00200000:2.0:1789760788.205045:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73818112 chunk: [73818112, 73822208) 73822208, 4096 00020000:00200000:2.0:1789760788.205050:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.205070:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18022, 18022] 00000008:00010000:2.0:1789760788.205088:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.205094:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.205106:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73818112, 73822208) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.205120:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73818112=0x4666000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.205174:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.205177:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73822208 00000020:00010000:2.0:1789760788.205186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.205190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.205193:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.205197:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.205200:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.205207:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.205213:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.205227:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760788.205342:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 73818112<>73822208 00000100:00100000:3.0:1789760788.205476:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565537664:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.212035:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985330 gen 3 00000008:00010000:3.0:1789760788.212087:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760788.212096:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.212108:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.212133:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760788.212164:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565537664:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.212291:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18022, 18022] 00000080:00200000:2.0:1789760788.212310:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.212316:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.212332:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73822208 i_blocks 144208 00000020:00200000:2.0:1789760788.212336:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.212346:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.212388:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73822208, count: 4096 00000080:00200000:2.0:1789760788.212396:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73822208, bytes: 4096 00000080:00200000:2.0:1789760788.212407:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.212415:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.212418:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73822208, 73826304) 0 0 00000080:00200000:2.0:1789760788.212423:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.212451:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18023, 18023] 00020000:00200000:2.0:1789760788.212463:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73822208 chunk: [73822208, 73826304) 73826304, 4096 00020000:00200000:2.0:1789760788.212468:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.212490:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18023, 18023] 00000008:00010000:2.0:1789760788.212508:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.212515:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.212528:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73822208, 73826304) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.212543:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73822208=0x4667000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.212605:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.212607:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73826304 00000020:00010000:2.0:1789760788.212616:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.212620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.212623:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.212627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.212630:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.212638:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.212696:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.212715:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760788.212928:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 73822208<>73826304 00000100:00100000:3.0:1789760788.213129:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565537792:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.235372:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985331 gen 3 00000008:00010000:3.0:1789760788.235421:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760788.235429:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.235449:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.235473:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760788.235607:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565537792:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.235773:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18023, 18023] 00000080:00200000:2.0:1789760788.235855:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.235861:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.235877:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73826304 i_blocks 144216 00000020:00200000:2.0:1789760788.235881:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.235891:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.235926:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73826304, count: 4096 00000080:00200000:2.0:1789760788.235934:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73826304, bytes: 4096 00000080:00200000:2.0:1789760788.235948:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.235957:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.235960:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73826304, 73830400) 0 0 00000080:00200000:2.0:1789760788.235964:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.235982:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18024, 18024] 00020000:00200000:2.0:1789760788.235989:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73826304 chunk: [73826304, 73830400) 73830400, 4096 00020000:00200000:2.0:1789760788.235994:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.236013:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18024, 18024] 00000008:00010000:2.0:1789760788.236030:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.236036:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.236047:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73826304, 73830400) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.236059:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73826304=0x4668000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.236119:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.236122:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73830400 00000020:00010000:2.0:1789760788.236130:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.236134:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.236137:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.236140:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.236143:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.236151:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.236159:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.236176:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760788.236299:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 73826304<>73830400 00000100:00100000:3.0:1789760788.236416:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565537920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.252578:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985332 gen 3 00000008:00010000:3.0:1789760788.252627:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760788.252636:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.252666:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.252693:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760788.252723:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565537920:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.253032:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18024, 18024] 00000080:00200000:2.0:1789760788.253050:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.253056:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.253073:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73830400 i_blocks 144224 00000020:00200000:2.0:1789760788.253077:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.253087:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.253122:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73830400, count: 4096 00000080:00200000:2.0:1789760788.253130:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73830400, bytes: 4096 00000080:00200000:2.0:1789760788.253140:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.253150:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.253153:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73830400, 73834496) 0 0 00000080:00200000:2.0:1789760788.253157:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.253176:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18025, 18025] 00020000:00200000:2.0:1789760788.253184:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73830400 chunk: [73830400, 73834496) 73834496, 4096 00020000:00200000:2.0:1789760788.253189:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.253209:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18025, 18025] 00000008:00010000:2.0:1789760788.253226:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.253232:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.253244:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73830400, 73834496) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.253259:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73830400=0x4669000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.253318:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.253321:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73834496 00000020:00010000:2.0:1789760788.253331:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.253334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.253337:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.253341:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.253344:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.253352:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.253359:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.253376:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760788.253502:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 73830400<>73834496 00000100:00100000:3.0:1789760788.253615:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565538048:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.263903:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985333 gen 3 00000008:00010000:3.0:1789760788.263948:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760788.263955:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.263966:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.263990:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760788.264021:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565538048:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.264123:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18025, 18025] 00000080:00200000:2.0:1789760788.264140:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.264147:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.264161:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73834496 i_blocks 144232 00000020:00200000:2.0:1789760788.264165:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.264175:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.264207:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73834496, count: 4096 00000080:00200000:2.0:1789760788.264213:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73834496, bytes: 4096 00000080:00200000:2.0:1789760788.264223:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.264231:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.264233:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73834496, 73838592) 0 0 00000080:00200000:2.0:1789760788.264238:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.264254:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18026, 18026] 00020000:00200000:2.0:1789760788.264478:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73834496 chunk: [73834496, 73838592) 73838592, 4096 00020000:00200000:2.0:1789760788.264540:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.264563:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18026, 18026] 00000008:00010000:2.0:1789760788.264582:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.264587:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.264598:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73834496, 73838592) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.264611:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73834496=0x466a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.264696:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.264698:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73838592 00000020:00010000:2.0:1789760788.264706:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.264709:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.264713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.264716:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.264719:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.264727:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.264733:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.264747:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189480 00000008:00100000:2.0:1789760788.265100:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 73834496<>73838592 00000100:00100000:3.0:1789760788.265214:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565538176:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.278617:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985334 gen 3 00000008:00010000:3.0:1789760788.278712:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189480 00000020:00200000:3.0:1789760788.278721:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.278734:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.278756:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189480 00000100:00100000:3.0:1789760788.278797:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565538176:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.279023:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18026, 18026] 00000080:00200000:2.0:1789760788.279039:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.279045:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.279059:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73838592 i_blocks 144240 00000020:00200000:2.0:1789760788.279063:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.279072:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.279106:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73838592, count: 4096 00000080:00200000:2.0:1789760788.279114:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73838592, bytes: 4096 00000080:00200000:2.0:1789760788.279122:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.279131:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.279133:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73838592, 73842688) 0 0 00000080:00200000:2.0:1789760788.279137:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.279153:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18027, 18027] 00020000:00200000:2.0:1789760788.279162:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73838592 chunk: [73838592, 73842688) 73842688, 4096 00020000:00200000:2.0:1789760788.279166:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.279184:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18027, 18027] 00000008:00010000:2.0:1789760788.279201:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.279207:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.279218:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73838592, 73842688) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.279231:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73838592=0x466b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.279284:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.279286:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73842688 00000020:00010000:2.0:1789760788.279294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.279298:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.279301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.279304:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.279308:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.279314:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.279320:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.279334:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760788.279465:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 73838592<>73842688 00000100:00100000:3.0:1789760788.279609:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565538304:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.293291:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985335 gen 3 00000008:00010000:3.0:1789760788.293343:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760788.293351:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.293362:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.293386:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000080:00200000:2.0:1789760788.293744:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18027, 18027] 00000080:00200000:2.0:1789760788.293764:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.293771:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.293859:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73842688 i_blocks 144248 00000020:00200000:2.0:1789760788.293864:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.293874:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.293906:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73842688, count: 4096 00000080:00200000:2.0:1789760788.293915:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73842688, bytes: 4096 00000080:00200000:2.0:1789760788.293926:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.293935:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.293938:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73842688, 73846784) 0 0 00000080:00200000:2.0:1789760788.293942:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.293958:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18028, 18028] 00020000:00200000:2.0:1789760788.293967:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73842688 chunk: [73842688, 73846784) 73846784, 4096 00020000:00200000:2.0:1789760788.293971:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.293993:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18028, 18028] 00000008:00010000:2.0:1789760788.294009:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.294015:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.294027:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73842688, 73846784) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.294041:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73842688=0x466c000, pages 1 (max 131072), lockless, parallel 00000100:00100000:3.0:1789760788.294330:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565538304:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760788.294381:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:3.0:1789760788.295345:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 73842688<>73846784 00000020:00200000:2.0:1789760788.295528:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.295537:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73846784 00000020:00010000:2.0:1789760788.295553:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.295558:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.295562:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.295566:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.295569:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.295580:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.295590:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:3.0:1789760788.297397:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565538432:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.310864:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985336 gen 3 00000008:00010000:2.0:1789760788.310911:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760788.310919:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.310930:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.310954:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:2.0:1789760788.310983:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565538432:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760788.311219:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18028, 18028] 00000080:00200000:0.0:1789760788.311235:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760788.311240:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760788.311257:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73846784 i_blocks 144256 00000020:00200000:0.0:1789760788.311260:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760788.311269:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760788.311302:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73846784, count: 4096 00000080:00200000:0.0:1789760788.311308:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73846784, bytes: 4096 00000080:00200000:0.0:1789760788.311318:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760788.311326:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760788.311328:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73846784, 73850880) 0 0 00000080:00200000:0.0:1789760788.311333:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760788.311347:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18029, 18029] 00020000:00200000:0.0:1789760788.311355:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73846784 chunk: [73846784, 73850880) 73850880, 4096 00020000:00200000:0.0:1789760788.311360:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760788.311387:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18029, 18029] 00000008:00010000:0.0:1789760788.311403:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760788.311409:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760788.311426:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73846784, 73850880) append 0 append lockpos 0 00000080:00200000:0.0:1789760788.311447:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73846784=0x466d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760788.311499:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760788.311501:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73850880 00000020:00010000:0.0:1789760788.311509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760788.311512:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760788.311515:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760788.311519:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760788.311522:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760788.311528:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760788.311534:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760788.311548:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760788.311692:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73846784<>73850880 00000100:00100000:0.0:1789760788.311726:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760788.311744:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565538560:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.329004:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985337 gen 3 00000008:00010000:0.0:1789760788.329050:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760788.329058:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.329068:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.329095:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760788.329124:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565538560:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.329304:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18029, 18029] 00000080:00200000:2.0:1789760788.329321:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.329328:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.329343:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73850880 i_blocks 144264 00000020:00200000:2.0:1789760788.329347:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.329359:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.329395:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73850880, count: 4096 00000080:00200000:2.0:1789760788.329411:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73850880, bytes: 4096 00000080:00200000:2.0:1789760788.329424:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.329433:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.329443:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73850880, 73854976) 0 0 00000080:00200000:2.0:1789760788.329447:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.329467:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18030, 18030] 00020000:00200000:2.0:1789760788.329477:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73850880 chunk: [73850880, 73854976) 73854976, 4096 00020000:00200000:2.0:1789760788.329482:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.329512:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18030, 18030] 00000008:00010000:2.0:1789760788.329530:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.329537:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.329548:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73850880, 73854976) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.329561:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73850880=0x466e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.329636:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.329640:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73854976 00000020:00010000:2.0:1789760788.329681:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.329686:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.329689:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.329692:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.329696:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.329704:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.329711:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.329726:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760788.329856:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 73850880<>73854976 00000100:00100000:2.0:1789760788.329901:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760788.329918:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565538688:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.348272:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985338 gen 3 00000008:00010000:2.0:1789760788.348323:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760788.348333:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.348345:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.348366:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:2.0:1789760788.348395:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565538688:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.348727:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18030, 18030] 00000080:00200000:3.0:1789760788.348744:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.348750:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.348764:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73854976 i_blocks 144272 00000020:00200000:3.0:1789760788.348769:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.348845:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.348881:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73854976, count: 4096 00000080:00200000:3.0:1789760788.348889:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73854976, bytes: 4096 00000080:00200000:3.0:1789760788.348900:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.348909:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.348911:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73854976, 73859072) 0 0 00000080:00200000:3.0:1789760788.348916:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.348933:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18031, 18031] 00020000:00200000:3.0:1789760788.348943:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73854976 chunk: [73854976, 73859072) 73859072, 4096 00020000:00200000:3.0:1789760788.348948:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.348968:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18031, 18031] 00000008:00010000:3.0:1789760788.348987:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.348993:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.349005:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73854976, 73859072) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.349019:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73854976=0x466f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.349081:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.349084:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73859072 00000020:00010000:3.0:1789760788.349093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.349097:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.349100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.349103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.349106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.349114:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.349121:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760788.349140:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760788.349264:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 73854976<>73859072 00000100:00100000:2.0:1789760788.349332:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565538816:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.362774:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985339 gen 3 00000008:00010000:2.0:1789760788.363917:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760788.363926:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.363939:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.363958:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760788.363992:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565538816:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.364823:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18031, 18031] 00000080:00200000:3.0:1789760788.364844:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.364850:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.364866:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73859072 i_blocks 144280 00000020:00200000:3.0:1789760788.364870:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.364881:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.364916:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73859072, count: 4096 00000080:00200000:3.0:1789760788.364923:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73859072, bytes: 4096 00000080:00200000:3.0:1789760788.364937:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.364946:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.364948:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73859072, 73863168) 0 0 00000080:00200000:3.0:1789760788.364953:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.364969:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18032, 18032] 00020000:00200000:3.0:1789760788.364978:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73859072 chunk: [73859072, 73863168) 73863168, 4096 00020000:00200000:3.0:1789760788.364983:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.365005:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18032, 18032] 00000008:00010000:3.0:1789760788.365023:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.365030:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.365041:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73859072, 73863168) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.365054:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73859072=0x4670000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.365114:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.365117:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73863168 00000020:00010000:3.0:1789760788.365125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.365129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.365132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.365136:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.365139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.365148:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.365156:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760788.365174:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760788.365313:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 73859072<>73863168 00000100:00100000:2.0:1789760788.365945:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565538944:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.374989:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985340 gen 3 00000008:00010000:2.0:1789760788.375049:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760788.375057:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.375068:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.375088:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760788.375117:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565538944:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.375280:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18032, 18032] 00000080:00200000:3.0:1789760788.375296:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.375302:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.375316:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73863168 i_blocks 144288 00000020:00200000:3.0:1789760788.375320:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.375329:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.375362:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73863168, count: 4096 00000080:00200000:3.0:1789760788.375369:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73863168, bytes: 4096 00000080:00200000:3.0:1789760788.375449:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.375458:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.375460:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73863168, 73867264) 0 0 00000080:00200000:3.0:1789760788.375466:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.375483:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18033, 18033] 00020000:00200000:3.0:1789760788.375602:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73863168 chunk: [73863168, 73867264) 73867264, 4096 00020000:00200000:3.0:1789760788.375609:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.375630:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18033, 18033] 00000008:00010000:3.0:1789760788.375698:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.375705:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.375718:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73863168, 73867264) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.375732:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73863168=0x4671000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.375788:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.375791:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73867264 00000020:00010000:3.0:1789760788.375799:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.375802:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.375805:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.375809:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.375812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.375821:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.375827:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760788.375844:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760788.375968:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 73863168<>73867264 00000100:00100000:2.0:1789760788.376042:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565539072:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.391256:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985341 gen 3 00000008:00010000:2.0:1789760788.391308:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760788.391316:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.391328:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.391352:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000080:00200000:3.0:1789760788.391476:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18033, 18033] 00000080:00200000:3.0:1789760788.391496:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.391502:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.391590:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73867264 i_blocks 144296 00000020:00200000:3.0:1789760788.391594:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.391606:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.391691:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73867264, count: 4096 00000080:00200000:3.0:1789760788.391702:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73867264, bytes: 4096 00000080:00200000:3.0:1789760788.391713:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.391722:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.391734:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73867264, 73871360) 0 0 00000080:00200000:3.0:1789760788.391738:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.391756:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18034, 18034] 00020000:00200000:3.0:1789760788.391765:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73867264 chunk: [73867264, 73871360) 73871360, 4096 00020000:00200000:3.0:1789760788.391770:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.391792:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18034, 18034] 00000008:00010000:3.0:1789760788.391809:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.391815:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.391828:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73867264, 73871360) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.391842:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73867264=0x4672000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.391900:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.391903:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73871360 00000020:00010000:3.0:1789760788.391912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.391915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.391918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.391922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.391924:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.391933:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.391941:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:2.0:1789760788.392112:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565539072:192.168.203.147@tcp:4:dd.0 00000008:00010000:2.0:1789760788.392160:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:2.0:1789760788.392282:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 73867264<>73871360 00000100:00100000:2.0:1789760788.392323:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760788.392337:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565539200:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.403996:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985342 gen 3 00000008:00010000:2.0:1789760788.404121:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760788.404131:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.404144:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.404175:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760788.404217:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565539200:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.404386:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18034, 18034] 00000080:00200000:3.0:1789760788.404405:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.404412:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.404432:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73871360 i_blocks 144304 00000020:00200000:3.0:1789760788.404445:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.404459:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.404864:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73871360, count: 4096 00000080:00200000:3.0:1789760788.404874:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73871360, bytes: 4096 00000080:00200000:2.0:1789760788.405076:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.405085:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.405088:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73871360, 73875456) 0 0 00000080:00200000:2.0:1789760788.405093:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.405110:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18035, 18035] 00020000:00200000:2.0:1789760788.405120:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73871360 chunk: [73871360, 73875456) 73875456, 4096 00020000:00200000:2.0:1789760788.405124:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.405147:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18035, 18035] 00000008:00010000:2.0:1789760788.405165:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.405171:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.405184:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73871360, 73875456) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.405199:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73871360=0x4673000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.405260:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.405263:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73875456 00000020:00010000:2.0:1789760788.405274:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.405277:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.405280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.405283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.405286:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.405295:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.405302:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.405354:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760788.406155:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 73871360<>73875456 00000100:00100000:3.0:1789760788.406252:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565539328:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.422087:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985343 gen 3 00000008:00010000:3.0:1789760788.422135:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760788.422144:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.422154:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.422178:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760788.422207:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565539328:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.422410:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18035, 18035] 00000080:00200000:2.0:1789760788.422427:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.422445:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.422461:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73875456 i_blocks 144312 00000020:00200000:2.0:1789760788.422465:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.422475:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.422509:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73875456, count: 4096 00000080:00200000:2.0:1789760788.422526:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73875456, bytes: 4096 00000080:00200000:2.0:1789760788.422538:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.422547:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.422550:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73875456, 73879552) 0 0 00000080:00200000:2.0:1789760788.422555:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.422571:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18036, 18036] 00020000:00200000:2.0:1789760788.422584:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73875456 chunk: [73875456, 73879552) 73879552, 4096 00020000:00200000:2.0:1789760788.422590:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.422618:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18036, 18036] 00000008:00010000:2.0:1789760788.422636:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.422686:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.422706:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73875456, 73879552) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.422722:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73875456=0x4674000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.422787:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.422790:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73879552 00000020:00010000:2.0:1789760788.422798:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.422801:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.422804:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.422807:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.422810:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.422818:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.422824:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.422841:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760788.422976:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 73875456<>73879552 00000100:00100000:3.0:1789760788.423788:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565539456:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.441548:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985344 gen 3 00000008:00010000:3.0:1789760788.441600:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760788.441609:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.441625:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.441685:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760788.441715:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565539456:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.443941:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18036, 18036] 00000080:00200000:2.0:1789760788.443967:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.443973:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.443990:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73879552 i_blocks 144320 00000020:00200000:2.0:1789760788.443996:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.444122:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.444156:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73879552, count: 4096 00000080:00200000:2.0:1789760788.444230:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73879552, bytes: 4096 00000080:00200000:2.0:1789760788.444312:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.444322:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.444325:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73879552, 73883648) 0 0 00000080:00200000:2.0:1789760788.444329:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.444347:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18037, 18037] 00020000:00200000:2.0:1789760788.444356:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73879552 chunk: [73879552, 73883648) 73883648, 4096 00020000:00200000:2.0:1789760788.444361:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.444701:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18037, 18037] 00000008:00010000:2.0:1789760788.444721:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.444728:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.444740:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73879552, 73883648) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.444754:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73879552=0x4675000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.444818:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.444821:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73883648 00000020:00010000:2.0:1789760788.444830:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.444833:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.444836:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.444839:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.444842:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.444851:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.444858:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.444881:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760788.445131:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 73879552<>73883648 00000100:00100000:3.0:1789760788.445869:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760788.445897:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565539584:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.459796:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985345 gen 3 00000008:00010000:3.0:1789760788.459845:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760788.459853:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.459864:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.459891:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760788.459923:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565539584:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.460285:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18037, 18037] 00000080:00200000:2.0:1789760788.460302:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.460308:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.460324:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73883648 i_blocks 144328 00000020:00200000:2.0:1789760788.460328:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.460338:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.460382:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73883648, count: 4096 00000080:00200000:2.0:1789760788.460391:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73883648, bytes: 4096 00000080:00200000:2.0:1789760788.460403:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.460412:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.460414:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73883648, 73887744) 0 0 00000080:00200000:2.0:1789760788.460419:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.460445:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18038, 18038] 00020000:00200000:2.0:1789760788.460454:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73883648 chunk: [73883648, 73887744) 73887744, 4096 00020000:00200000:2.0:1789760788.460459:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.460480:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18038, 18038] 00000008:00010000:2.0:1789760788.460498:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.460504:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.460516:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73883648, 73887744) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.460635:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73883648=0x4676000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.460730:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.460734:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73887744 00000020:00010000:2.0:1789760788.460743:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.460746:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.460749:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.460752:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.460755:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.460763:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.460770:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.460842:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760788.461257:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 73883648<>73887744 00000100:00100000:3.0:1789760788.461356:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565539712:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.481154:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985346 gen 3 00000008:00010000:3.0:1789760788.481206:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760788.481215:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.481238:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.481264:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760788.481296:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565539712:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760788.481862:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18038, 18038] 00000080:00200000:0.0:1789760788.481880:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760788.481886:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760788.481901:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73887744 i_blocks 144336 00000020:00200000:0.0:1789760788.481905:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760788.481915:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760788.481952:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73887744, count: 4096 00000080:00200000:0.0:1789760788.481960:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73887744, bytes: 4096 00000080:00200000:0.0:1789760788.481973:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760788.481983:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760788.481986:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73887744, 73891840) 0 0 00000080:00200000:0.0:1789760788.481991:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760788.482009:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18039, 18039] 00020000:00200000:0.0:1789760788.482017:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73887744 chunk: [73887744, 73891840) 73891840, 4096 00020000:00200000:0.0:1789760788.482021:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760788.482041:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18039, 18039] 00000008:00010000:0.0:1789760788.482059:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760788.482064:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760788.482075:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73887744, 73891840) append 0 append lockpos 0 00000080:00200000:0.0:1789760788.482099:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73887744=0x4677000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760788.482159:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760788.482162:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73891840 00000020:00010000:0.0:1789760788.482170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760788.482173:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760788.482176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760788.482181:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760788.482184:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760788.482191:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760788.482198:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760788.482215:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760788.482607:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73887744<>73891840 00000100:00100000:1.0:1789760788.483636:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760788.490842:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565539840:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.502706:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985347 gen 3 00000008:00010000:0.0:1789760788.502756:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760788.502766:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.502777:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.502788:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760788.502816:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565539840:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.503017:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18039, 18039] 00000080:00200000:1.0:1789760788.503033:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.503039:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.503054:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73891840 i_blocks 144344 00000020:00200000:1.0:1789760788.503058:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.503070:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.503116:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73891840, count: 4096 00000080:00200000:1.0:1789760788.503124:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73891840, bytes: 4096 00000080:00200000:1.0:1789760788.503134:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.503143:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.503146:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73891840, 73895936) 0 0 00000080:00200000:1.0:1789760788.503151:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.503169:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18040, 18040] 00020000:00200000:1.0:1789760788.503178:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73891840 chunk: [73891840, 73895936) 73895936, 4096 00020000:00200000:1.0:1789760788.503183:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.503205:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18040, 18040] 00000008:00010000:1.0:1789760788.503238:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.503244:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.503256:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73891840, 73895936) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.503270:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73891840=0x4678000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.503328:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.503331:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73895936 00000020:00010000:1.0:1789760788.503339:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.503342:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.503345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.503349:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.503352:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.503360:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.503367:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.503384:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760788.503516:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 73891840<>73895936 00000100:00100000:0.0:1789760788.503938:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760788.503956:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565539968:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.518498:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985348 gen 3 00000008:00010000:0.0:1789760788.518619:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760788.518628:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.518640:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.518733:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760788.518767:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565539968:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.518861:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18040, 18040] 00000080:00200000:1.0:1789760788.518878:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.518884:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.518898:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73895936 i_blocks 144352 00000020:00200000:1.0:1789760788.518902:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.519213:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.519310:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73895936, count: 4096 00000080:00200000:1.0:1789760788.519319:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73895936, bytes: 4096 00000080:00200000:1.0:1789760788.519334:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.519344:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.519347:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73895936, 73900032) 0 0 00000080:00200000:1.0:1789760788.519351:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.519368:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18041, 18041] 00020000:00200000:1.0:1789760788.519378:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73895936 chunk: [73895936, 73900032) 73900032, 4096 00020000:00200000:1.0:1789760788.519383:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.519405:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18041, 18041] 00000008:00010000:1.0:1789760788.519424:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.519431:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.519453:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73895936, 73900032) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.519467:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73895936=0x4679000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.519538:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.519541:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73900032 00000020:00010000:1.0:1789760788.519550:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.519554:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.519557:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.519560:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.519563:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.519572:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.519579:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.519602:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760788.519757:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 73895936<>73900032 00000100:00100000:0.0:1789760788.519849:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565540096:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.535028:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985349 gen 3 00000008:00010000:0.0:1789760788.535081:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760788.535090:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.535101:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.535172:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760788.535206:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565540096:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.541157:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18041, 18041] 00000080:00200000:1.0:1789760788.541187:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.541195:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.541266:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73900032 i_blocks 144360 00000020:00200000:1.0:1789760788.541273:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.541290:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.541329:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73900032, count: 4096 00000080:00200000:1.0:1789760788.541338:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73900032, bytes: 4096 00000080:00200000:1.0:1789760788.541353:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.541363:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.541365:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73900032, 73904128) 0 0 00000080:00200000:1.0:1789760788.541498:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.541581:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18042, 18042] 00020000:00200000:1.0:1789760788.541592:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73900032 chunk: [73900032, 73904128) 73904128, 4096 00020000:00200000:1.0:1789760788.541597:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.541817:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18042, 18042] 00000008:00010000:1.0:1789760788.541837:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.541843:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.541856:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73900032, 73904128) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.541872:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73900032=0x467a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.542002:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.542006:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73904128 00000020:00010000:1.0:1789760788.542016:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.542019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.542023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.542026:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.542028:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.542037:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.542044:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.542059:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760788.542193:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 73900032<>73904128 00000100:00100000:0.0:1789760788.542290:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565540224:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.560784:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985350 gen 3 00000008:00010000:0.0:1789760788.560839:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760788.560848:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.560860:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.560873:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760788.560901:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565540224:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.563202:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18042, 18042] 00000080:00200000:1.0:1789760788.563237:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.563244:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.563263:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73904128 i_blocks 144368 00000020:00200000:1.0:1789760788.563268:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.563285:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.563321:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73904128, count: 4096 00000080:00200000:1.0:1789760788.563330:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73904128, bytes: 4096 00000080:00200000:1.0:1789760788.563346:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.563355:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.563357:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73904128, 73908224) 0 0 00000080:00200000:1.0:1789760788.563362:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.563447:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18043, 18043] 00020000:00200000:1.0:1789760788.563459:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73904128 chunk: [73904128, 73908224) 73908224, 4096 00020000:00200000:1.0:1789760788.563463:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.563487:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18043, 18043] 00000008:00010000:1.0:1789760788.563508:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.563515:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.563536:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73904128, 73908224) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.563551:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73904128=0x467b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.563626:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.563629:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73908224 00000020:00010000:1.0:1789760788.563637:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.563707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.563712:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.563716:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.563719:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.563729:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.563736:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.563760:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760788.563894:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 73904128<>73908224 00000100:00100000:0.0:1789760788.565347:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565540352:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.579233:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985351 gen 3 00000008:00010000:0.0:1789760788.579290:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760788.579298:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.579309:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.579406:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760788.579447:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565540352:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.579979:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18043, 18043] 00000080:00200000:2.0:1789760788.579997:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.580004:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.580020:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73908224 i_blocks 144376 00000020:00200000:2.0:1789760788.580024:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.580036:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.580072:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73908224, count: 4096 00000080:00200000:2.0:1789760788.580080:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73908224, bytes: 4096 00000080:00200000:2.0:1789760788.580091:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.580101:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.580103:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73908224, 73912320) 0 0 00000080:00200000:2.0:1789760788.580108:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.580124:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18044, 18044] 00020000:00200000:2.0:1789760788.580133:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73908224 chunk: [73908224, 73912320) 73912320, 4096 00020000:00200000:2.0:1789760788.580137:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.580156:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18044, 18044] 00000008:00010000:2.0:1789760788.580172:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.580178:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.580189:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73908224, 73912320) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.580202:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73908224=0x467c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.580323:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.580327:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73912320 00000020:00010000:2.0:1789760788.580336:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.580339:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.580343:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.580346:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.580348:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.580355:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.580361:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.580448:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760788.580564:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 73908224<>73912320 00000100:00100000:3.0:1789760788.580657:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565540480:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.593283:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985352 gen 3 00000008:00010000:2.0:1789760788.593330:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:2.0:1789760788.593339:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.593350:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.593493:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:2.0:1789760788.593524:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565540480:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760788.597074:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18044, 18044] 00000080:00200000:0.0:1789760788.597102:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760788.597109:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760788.597127:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73912320 i_blocks 144384 00000020:00200000:0.0:1789760788.597134:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760788.597150:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760788.597184:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73912320, count: 4096 00000080:00200000:0.0:1789760788.597193:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73912320, bytes: 4096 00000080:00200000:0.0:1789760788.597207:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760788.597216:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760788.597219:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73912320, 73916416) 0 0 00000080:00200000:0.0:1789760788.597223:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760788.597241:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18045, 18045] 00020000:00200000:0.0:1789760788.597250:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73912320 chunk: [73912320, 73916416) 73916416, 4096 00020000:00200000:0.0:1789760788.597255:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760788.597278:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18045, 18045] 00000008:00010000:0.0:1789760788.597296:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760788.597302:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760788.597314:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73912320, 73916416) append 0 append lockpos 0 00000080:00200000:0.0:1789760788.597329:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73912320=0x467d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760788.597476:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760788.597480:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73916416 00000020:00010000:0.0:1789760788.597551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760788.597556:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760788.597559:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760788.597562:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760788.597566:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760788.597575:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760788.597582:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760788.597611:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760788.597776:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 73912320<>73916416 00000100:00100000:0.0:1789760788.597824:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760788.597849:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565540608:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.614628:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985353 gen 3 00000008:00010000:0.0:1789760788.614734:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:0.0:1789760788.614744:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.614754:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.614766:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:0.0:1789760788.614811:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565540608:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.614955:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18045, 18045] 00000080:00200000:1.0:1789760788.614972:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.614978:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.614993:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73916416 i_blocks 144392 00000020:00200000:1.0:1789760788.614997:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.615009:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.615048:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73916416, count: 4096 00000080:00200000:1.0:1789760788.615055:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73916416, bytes: 4096 00000080:00200000:1.0:1789760788.615066:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.615074:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.615077:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73916416, 73920512) 0 0 00000080:00200000:1.0:1789760788.615082:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.615101:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18046, 18046] 00020000:00200000:1.0:1789760788.615109:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73916416 chunk: [73916416, 73920512) 73920512, 4096 00020000:00200000:1.0:1789760788.615114:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.615134:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18046, 18046] 00000008:00010000:1.0:1789760788.615151:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.615157:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.615168:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73916416, 73920512) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.615183:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73916416=0x467e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.615242:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.615245:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73920512 00000020:00010000:1.0:1789760788.615254:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.615258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.615261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.615265:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.615267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.615275:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.615282:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.615299:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760788.615448:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 73916416<>73920512 00000100:00100000:1.0:1789760788.615503:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760788.615520:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565540736:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.629890:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985354 gen 3 00000008:00010000:0.0:1789760788.629953:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760788.629961:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.629974:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.629996:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760788.630024:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565540736:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.630178:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18046, 18046] 00000080:00200000:1.0:1789760788.630194:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.630201:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.630215:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73920512 i_blocks 144400 00000020:00200000:1.0:1789760788.630219:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.630230:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.630267:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73920512, count: 4096 00000080:00200000:1.0:1789760788.630275:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73920512, bytes: 4096 00000080:00200000:1.0:1789760788.630284:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.630292:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.630295:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73920512, 73924608) 0 0 00000080:00200000:1.0:1789760788.630299:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.630317:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18047, 18047] 00020000:00200000:1.0:1789760788.630326:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73920512 chunk: [73920512, 73924608) 73924608, 4096 00020000:00200000:1.0:1789760788.630331:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.630351:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18047, 18047] 00000008:00010000:1.0:1789760788.630378:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.630384:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.630397:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73920512, 73924608) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.630411:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73920512=0x467f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.630480:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.630483:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73924608 00000020:00010000:1.0:1789760788.630492:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.630495:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.630499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.630503:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.630506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.630514:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.630521:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.630537:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760788.630697:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73920512<>73924608 00000100:00100000:0.0:1789760788.630894:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565540864:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.645097:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985355 gen 3 00000008:00010000:0.0:1789760788.645150:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760788.645159:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.645170:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.645245:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760788.645278:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565540864:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.648501:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18047, 18047] 00000080:00200000:1.0:1789760788.648528:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.648535:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.648637:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73924608 i_blocks 144408 00000020:00200000:1.0:1789760788.648711:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.648728:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.648762:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73924608, count: 4096 00000080:00200000:1.0:1789760788.648772:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73924608, bytes: 4096 00000080:00200000:1.0:1789760788.648788:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.648797:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.648799:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73924608, 73928704) 0 0 00000080:00200000:1.0:1789760788.648804:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.648821:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18048, 18048] 00020000:00200000:1.0:1789760788.648830:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73924608 chunk: [73924608, 73928704) 73928704, 4096 00020000:00200000:1.0:1789760788.648834:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.648858:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18048, 18048] 00000008:00010000:1.0:1789760788.648876:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.648882:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.648972:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73924608, 73928704) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.648989:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73924608=0x4680000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.649055:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.649058:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73928704 00000020:00010000:1.0:1789760788.649067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.649070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.649084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.649088:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.649091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.649099:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.649107:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.649125:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760788.649261:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73924608<>73928704 00000100:00100000:0.0:1789760788.649353:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565540992:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.661510:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985356 gen 3 00000008:00010000:0.0:1789760788.661775:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760788.661784:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.661795:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.661860:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760788.661894:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565540992:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.661984:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18048, 18048] 00000080:00200000:1.0:1789760788.662070:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.662150:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.662165:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73928704 i_blocks 144416 00000020:00200000:1.0:1789760788.662169:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.662179:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.662218:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73928704, count: 4096 00000080:00200000:1.0:1789760788.662226:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73928704, bytes: 4096 00000080:00200000:1.0:1789760788.662237:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.662247:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.662250:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73928704, 73932800) 0 0 00000080:00200000:1.0:1789760788.662255:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.662272:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18049, 18049] 00020000:00200000:1.0:1789760788.662282:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73928704 chunk: [73928704, 73932800) 73932800, 4096 00020000:00200000:1.0:1789760788.662287:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.662305:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18049, 18049] 00000008:00010000:1.0:1789760788.662322:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.662328:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.662338:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73928704, 73932800) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.662351:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73928704=0x4681000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.662460:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.662463:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73932800 00000020:00010000:1.0:1789760788.662471:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.662474:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.662477:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.662482:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.662484:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.662491:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.662498:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.662515:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760788.662770:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 73928704<>73932800 00000100:00100000:0.0:1789760788.662840:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565541120:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.680108:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985357 gen 3 00000008:00010000:0.0:1789760788.680157:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760788.680165:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.680177:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.680197:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760788.680226:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565541120:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.680408:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18049, 18049] 00000080:00200000:1.0:1789760788.680424:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.680444:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.680460:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73932800 i_blocks 144424 00000020:00200000:1.0:1789760788.680463:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.680474:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.680510:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73932800, count: 4096 00000080:00200000:1.0:1789760788.680518:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73932800, bytes: 4096 00000080:00200000:1.0:1789760788.680528:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.680547:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.680550:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73932800, 73936896) 0 0 00000080:00200000:1.0:1789760788.680555:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.680571:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18050, 18050] 00020000:00200000:1.0:1789760788.680580:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73932800 chunk: [73932800, 73936896) 73936896, 4096 00020000:00200000:1.0:1789760788.680585:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.680604:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18050, 18050] 00000008:00010000:1.0:1789760788.680621:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.680626:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.680637:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73932800, 73936896) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.680684:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73932800=0x4682000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.680743:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.680746:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73936896 00000020:00010000:1.0:1789760788.680755:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.680759:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.680762:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.680767:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.680770:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.680778:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.680785:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.680802:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760788.680923:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 73932800<>73936896 00000100:00100000:0.0:1789760788.681089:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760788.681108:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565541248:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.689342:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985358 gen 3 00000008:00010000:0.0:1789760788.689403:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760788.689411:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.689422:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.689449:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760788.689481:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565541248:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.689696:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18050, 18050] 00000080:00200000:1.0:1789760788.689715:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.689722:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.689739:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73936896 i_blocks 144432 00000020:00200000:1.0:1789760788.689743:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.689754:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.689792:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73936896, count: 4096 00000080:00200000:1.0:1789760788.689800:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73936896, bytes: 4096 00000080:00200000:1.0:1789760788.689810:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.689819:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.689822:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73936896, 73940992) 0 0 00000080:00200000:1.0:1789760788.689826:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.689844:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18051, 18051] 00020000:00200000:1.0:1789760788.689853:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73936896 chunk: [73936896, 73940992) 73940992, 4096 00020000:00200000:1.0:1789760788.689858:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.689878:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18051, 18051] 00000008:00010000:1.0:1789760788.689898:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.690365:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.690453:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73936896, 73940992) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.690469:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73936896=0x4683000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.690535:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.690539:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73940992 00000020:00010000:1.0:1789760788.690547:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.690551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.690554:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.690559:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.690562:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.690572:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.690579:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.690605:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760788.690765:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 73936896<>73940992 00000100:00100000:0.0:1789760788.692137:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565541376:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.716178:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985359 gen 3 00000008:00010000:0.0:1789760788.716257:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760788.716266:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.716277:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.716292:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760788.716317:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565541376:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.716450:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18051, 18051] 00000080:00200000:1.0:1789760788.716466:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.716472:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.716486:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73940992 i_blocks 144440 00000020:00200000:1.0:1789760788.716490:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.716499:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.716533:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73940992, count: 4096 00000080:00200000:1.0:1789760788.716541:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73940992, bytes: 4096 00000080:00200000:1.0:1789760788.716555:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.716564:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.716566:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73940992, 73945088) 0 0 00000080:00200000:1.0:1789760788.716570:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.716588:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18052, 18052] 00020000:00200000:1.0:1789760788.716597:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73940992 chunk: [73940992, 73945088) 73945088, 4096 00020000:00200000:1.0:1789760788.716602:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.716621:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18052, 18052] 00000008:00010000:1.0:1789760788.716638:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.716671:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.716684:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73940992, 73945088) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.716698:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73940992=0x4684000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.716753:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.716756:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73945088 00000020:00010000:1.0:1789760788.716765:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.716769:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.716772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.716777:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.716780:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.716788:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.716795:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.716812:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760788.717680:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 73940992<>73945088 00000100:00100000:0.0:1789760788.717789:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760788.717810:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565541504:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.734722:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985360 gen 3 00000008:00010000:0.0:1789760788.735087:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760788.735097:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.735110:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.735135:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760788.735169:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565541504:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.736033:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18052, 18052] 00000080:00200000:1.0:1789760788.736054:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.736060:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.736079:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73945088 i_blocks 144448 00000020:00200000:1.0:1789760788.736084:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.736096:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.736137:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73945088, count: 4096 00000080:00200000:1.0:1789760788.736378:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73945088, bytes: 4096 00000080:00200000:1.0:1789760788.736397:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.736406:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.736408:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73945088, 73949184) 0 0 00000080:00200000:1.0:1789760788.736413:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.736429:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18053, 18053] 00020000:00200000:1.0:1789760788.736448:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73945088 chunk: [73945088, 73949184) 73949184, 4096 00020000:00200000:1.0:1789760788.736453:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.736474:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18053, 18053] 00000008:00010000:1.0:1789760788.736492:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.736498:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.736509:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73945088, 73949184) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.736523:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73945088=0x4685000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.736587:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.736591:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73949184 00000020:00010000:1.0:1789760788.736599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.736603:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.736606:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.736611:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.736614:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.736622:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.736629:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.738068:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760788.738218:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 73945088<>73949184 00000100:00100000:0.0:1789760788.739360:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760788.739395:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565541632:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.754356:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985361 gen 3 00000008:00010000:0.0:1789760788.754409:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760788.754417:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.754429:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.756548:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760788.756601:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565541632:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.759089:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18053, 18053] 00000080:00200000:2.0:1789760788.759119:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.759127:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.759146:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73949184 i_blocks 144456 00000020:00200000:2.0:1789760788.759152:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.759169:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.759205:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73949184, count: 4096 00000080:00200000:2.0:1789760788.759215:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73949184, bytes: 4096 00000080:00200000:2.0:1789760788.759230:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.759239:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.759241:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73949184, 73953280) 0 0 00000080:00200000:2.0:1789760788.759246:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.759263:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18054, 18054] 00020000:00200000:2.0:1789760788.759272:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73949184 chunk: [73949184, 73953280) 73953280, 4096 00020000:00200000:2.0:1789760788.759277:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.759299:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18054, 18054] 00000008:00010000:2.0:1789760788.759318:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.759325:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.759337:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73949184, 73953280) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.759351:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73949184=0x4686000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.759421:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.759425:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73953280 00000020:00010000:2.0:1789760788.759444:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.759449:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.759452:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ac68/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.759456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.759460:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.759469:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.760053:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.760087:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189240 00000008:00100000:2.0:1789760788.761099:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 73949184<>73953280 00000100:00100000:3.0:1789760788.761192:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760788.765076:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565541760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.777473:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985362 gen 3 00000008:00010000:3.0:1789760788.777590:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189240 00000020:00200000:3.0:1789760788.777599:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.777611:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.777738:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189240 00000100:00100000:3.0:1789760788.777773:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565541760:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760788.781872:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18054, 18054] 00000080:00200000:0.0:1789760788.781901:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760788.782173:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.782875:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73953280 i_blocks 144464 00000020:00200000:2.0:1789760788.782885:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.782903:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.782943:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73953280, count: 4096 00000080:00200000:2.0:1789760788.782952:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73953280, bytes: 4096 00000080:00200000:2.0:1789760788.782967:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.782977:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.782979:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73953280, 73957376) 0 0 00000080:00200000:2.0:1789760788.782985:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.783002:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18055, 18055] 00020000:00200000:2.0:1789760788.783012:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73953280 chunk: [73953280, 73957376) 73957376, 4096 00020000:00200000:2.0:1789760788.783017:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.783041:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18055, 18055] 00000008:00010000:2.0:1789760788.783062:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.783068:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.783081:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73953280, 73957376) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.783095:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73953280=0x4687000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.783165:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.783168:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73957376 00000020:00010000:2.0:1789760788.783176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.783180:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.783183:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.783187:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.783190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.783198:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.783205:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.783258:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760788.785578:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 73953280<>73957376 00000100:00100000:2.0:1789760788.785698:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760788.785728:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565541888:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.806071:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985363 gen 3 00000008:00010000:2.0:1789760788.806110:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760788.806117:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.806125:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.806145:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760788.806166:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565541888:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760788.806254:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18055, 18055] 00000080:00200000:3.0:1789760788.806267:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760788.806272:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760788.806284:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73957376 i_blocks 144472 00000020:00200000:3.0:1789760788.806287:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760788.806296:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760788.806341:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73957376, count: 4096 00000080:00200000:3.0:1789760788.806348:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73957376, bytes: 4096 00000080:00200000:3.0:1789760788.806356:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760788.806363:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760788.806365:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73957376, 73961472) 0 0 00000080:00200000:3.0:1789760788.806379:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760788.806392:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18056, 18056] 00020000:00200000:3.0:1789760788.806398:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73957376 chunk: [73957376, 73961472) 73961472, 4096 00020000:00200000:3.0:1789760788.806402:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760788.806416:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18056, 18056] 00000008:00010000:3.0:1789760788.806428:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760788.806442:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760788.806451:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73957376, 73961472) append 0 append lockpos 0 00000080:00200000:3.0:1789760788.806477:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73957376=0x4688000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760788.806519:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760788.806522:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73961472 00000020:00010000:3.0:1789760788.806528:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760788.806532:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.806534:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760788.806537:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760788.806540:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760788.806547:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760788.806553:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760788.806567:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760788.806729:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 73957376<>73961472 00000100:00100000:2.0:1789760788.806925:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565542016:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760788.820084:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985364 gen 3 00000008:00010000:2.0:1789760788.820124:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760788.820131:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760788.820138:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760788.820162:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760788.820187:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565542016:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.820298:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18056, 18056] 00000080:00200000:1.0:1789760788.820310:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.820315:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.820326:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73961472 i_blocks 144480 00000020:00200000:1.0:1789760788.820329:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.820337:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.820359:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73961472, count: 4096 00000080:00200000:1.0:1789760788.820366:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73961472, bytes: 4096 00000080:00200000:1.0:1789760788.820448:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.820455:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.820458:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73961472, 73965568) 0 0 00000080:00200000:1.0:1789760788.820769:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.820784:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18057, 18057] 00020000:00200000:1.0:1789760788.820791:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73961472 chunk: [73961472, 73965568) 73965568, 4096 00020000:00200000:1.0:1789760788.820795:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.820809:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18057, 18057] 00000008:00010000:1.0:1789760788.820926:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.820932:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.820943:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73961472, 73965568) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.821097:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73961472=0x4689000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.821146:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.821149:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73965568 00000020:00010000:1.0:1789760788.821155:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.821158:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.821161:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.821164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.821166:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.821173:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.821178:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.821195:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760788.821301:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 73961472<>73965568 00000100:00100000:0.0:1789760788.821389:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565542144:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.844206:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985365 gen 3 00000008:00010000:0.0:1789760788.844239:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760788.844245:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.844252:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.844268:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760788.844287:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565542144:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.844565:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18057, 18057] 00000080:00200000:1.0:1789760788.844578:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.844583:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.844594:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73965568 i_blocks 144488 00000020:00200000:1.0:1789760788.844598:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.844605:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.844628:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73965568, count: 4096 00000080:00200000:1.0:1789760788.844635:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73965568, bytes: 4096 00000080:00200000:1.0:1789760788.844669:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.844676:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.844678:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73965568, 73969664) 0 0 00000080:00200000:1.0:1789760788.844682:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.844694:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18058, 18058] 00020000:00200000:1.0:1789760788.844701:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73965568 chunk: [73965568, 73969664) 73969664, 4096 00020000:00200000:1.0:1789760788.844705:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.844719:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18058, 18058] 00000008:00010000:1.0:1789760788.844730:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.844735:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.844742:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73965568, 73969664) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.844760:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73965568=0x468a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.845623:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.845628:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73969664 00000020:00010000:1.0:1789760788.845636:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.845640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.845662:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.845666:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.845669:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.845676:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.845681:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.845694:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760788.845904:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 73965568<>73969664 00000100:00100000:0.0:1789760788.846093:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565542272:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.860102:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985366 gen 3 00000008:00010000:0.0:1789760788.860138:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760788.860144:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.860153:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.860178:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760788.860202:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565542272:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.860482:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18058, 18058] 00000080:00200000:1.0:1789760788.860494:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.860500:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.860511:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73969664 i_blocks 144496 00000020:00200000:1.0:1789760788.860514:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.860522:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.860545:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73969664, count: 4096 00000080:00200000:1.0:1789760788.860551:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73969664, bytes: 4096 00000080:00200000:1.0:1789760788.860559:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.860566:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.860568:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73969664, 73973760) 0 0 00000080:00200000:1.0:1789760788.860572:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.860585:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18059, 18059] 00020000:00200000:1.0:1789760788.860593:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73969664 chunk: [73969664, 73973760) 73973760, 4096 00020000:00200000:1.0:1789760788.860596:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.860610:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18059, 18059] 00000008:00010000:1.0:1789760788.860623:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.860628:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.860638:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73969664, 73973760) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.860685:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73969664=0x468b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.860742:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.860745:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73973760 00000020:00010000:1.0:1789760788.860760:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.860764:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.860767:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.860770:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.860773:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.860780:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.860786:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.860798:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760788.860885:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 73969664<>73973760 00000100:00100000:0.0:1789760788.861181:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760788.861199:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565542400:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.873760:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985367 gen 3 00000008:00010000:0.0:1789760788.873823:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760788.873832:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.873843:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.873868:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760788.873899:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565542400:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.874284:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18059, 18059] 00000080:00200000:1.0:1789760788.874301:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.874307:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.874322:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73973760 i_blocks 144504 00000020:00200000:1.0:1789760788.874326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.874336:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.874385:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73973760, count: 4096 00000080:00200000:1.0:1789760788.874393:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73973760, bytes: 4096 00000080:00200000:1.0:1789760788.874408:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.874416:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.874419:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73973760, 73977856) 0 0 00000080:00200000:1.0:1789760788.874423:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.874452:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18060, 18060] 00020000:00200000:1.0:1789760788.874461:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73973760 chunk: [73973760, 73977856) 73977856, 4096 00020000:00200000:1.0:1789760788.874466:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.874497:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18060, 18060] 00000008:00010000:1.0:1789760788.874518:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.874524:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.874537:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73973760, 73977856) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.874553:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73973760=0x468c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.874615:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.874618:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73977856 00000020:00010000:1.0:1789760788.874627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.874631:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.874634:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.874638:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.874641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.874699:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.874706:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.874725:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760788.874859:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 73973760<>73977856 00000100:00100000:0.0:1789760788.875183:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565542528:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.885036:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985368 gen 3 00000008:00010000:0.0:1789760788.885087:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760788.885095:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.885107:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.885130:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760788.885158:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565542528:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.885271:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18060, 18060] 00000080:00200000:1.0:1789760788.885289:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.885296:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.885311:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73977856 i_blocks 144512 00000020:00200000:1.0:1789760788.885316:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.885326:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.885362:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73977856, count: 4096 00000080:00200000:1.0:1789760788.885380:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73977856, bytes: 4096 00000080:00200000:1.0:1789760788.885391:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.885400:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.885403:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73977856, 73981952) 0 0 00000080:00200000:1.0:1789760788.885408:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.885424:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18061, 18061] 00020000:00200000:1.0:1789760788.885441:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73977856 chunk: [73977856, 73981952) 73981952, 4096 00020000:00200000:1.0:1789760788.885446:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.885467:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18061, 18061] 00000008:00010000:1.0:1789760788.885501:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.885508:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.885520:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73977856, 73981952) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.885534:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73977856=0x468d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.885591:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.885594:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73981952 00000020:00010000:1.0:1789760788.885602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.885605:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.885609:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.885613:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.885616:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.885624:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.885631:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.885685:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760788.885809:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 73977856<>73981952 00000100:00100000:0.0:1789760788.886141:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565542656:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.895103:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985369 gen 3 00000008:00010000:0.0:1789760788.895157:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760788.895165:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.895177:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.895387:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760788.895419:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565542656:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.896049:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18061, 18061] 00000080:00200000:1.0:1789760788.896066:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.896072:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.896089:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73981952 i_blocks 144520 00000020:00200000:1.0:1789760788.896093:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.896106:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.896143:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73981952, count: 4096 00000080:00200000:1.0:1789760788.896151:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73981952, bytes: 4096 00000080:00200000:1.0:1789760788.896163:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.896172:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.896174:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73981952, 73986048) 0 0 00000080:00200000:1.0:1789760788.896178:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.896194:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18062, 18062] 00020000:00200000:1.0:1789760788.896204:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73981952 chunk: [73981952, 73986048) 73986048, 4096 00020000:00200000:1.0:1789760788.896209:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.896232:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18062, 18062] 00000008:00010000:1.0:1789760788.896251:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.896258:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.896271:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73981952, 73986048) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.896284:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73981952=0x468e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.896342:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.896345:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73986048 00000020:00010000:1.0:1789760788.896354:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.896358:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.896361:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.896365:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.896379:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.896388:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.896395:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.896415:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760788.896555:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 73981952<>73986048 00000100:00100000:0.0:1789760788.896795:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565542784:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.908822:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985370 gen 3 00000008:00010000:0.0:1789760788.908873:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760788.908882:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.908893:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.908921:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760788.909083:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565542784:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.909254:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18062, 18062] 00000080:00200000:1.0:1789760788.909270:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.909276:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.909292:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73986048 i_blocks 144528 00000020:00200000:1.0:1789760788.909296:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.909307:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.909345:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73986048, count: 4096 00000080:00200000:1.0:1789760788.909353:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73986048, bytes: 4096 00000080:00200000:1.0:1789760788.909362:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.909372:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.909374:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73986048, 73990144) 0 0 00000080:00200000:1.0:1789760788.909378:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.909395:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18063, 18063] 00020000:00200000:1.0:1789760788.909403:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73986048 chunk: [73986048, 73990144) 73990144, 4096 00020000:00200000:1.0:1789760788.909408:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.909428:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18063, 18063] 00000008:00010000:1.0:1789760788.909458:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.909465:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.909477:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73986048, 73990144) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.909500:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73986048=0x468f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.909560:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.909563:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73990144 00000020:00010000:1.0:1789760788.909570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.909574:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.909578:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.909582:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.909585:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.909592:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.909599:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.909617:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760788.909774:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 73986048<>73990144 00000100:00100000:0.0:1789760788.909920:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565542912:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.923736:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985371 gen 3 00000008:00010000:0.0:1789760788.923790:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760788.923799:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.923811:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.923838:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760788.923870:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565542912:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.924404:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18063, 18063] 00000080:00200000:1.0:1789760788.924424:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.924438:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.924456:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73990144 i_blocks 144536 00000020:00200000:1.0:1789760788.924460:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.924471:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.924508:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73990144, count: 4096 00000080:00200000:1.0:1789760788.924516:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73990144, bytes: 4096 00000080:00200000:1.0:1789760788.924529:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.924538:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.924541:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73990144, 73994240) 0 0 00000080:00200000:1.0:1789760788.924546:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.924564:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18064, 18064] 00020000:00200000:1.0:1789760788.924586:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73990144 chunk: [73990144, 73994240) 73994240, 4096 00020000:00200000:1.0:1789760788.924591:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.924613:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18064, 18064] 00000008:00010000:1.0:1789760788.924632:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.924639:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.924687:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73990144, 73994240) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.924702:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73990144=0x4690000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.924762:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.924766:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73994240 00000020:00010000:1.0:1789760788.924774:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.924777:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.924780:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.924783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.924786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.924794:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.924803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.924823:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760788.924971:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 73990144<>73994240 00000100:00100000:0.0:1789760788.925260:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565543040:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.942154:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985372 gen 3 00000008:00010000:0.0:1789760788.942204:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760788.942212:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.942223:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.942244:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760788.942273:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565543040:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.942877:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18064, 18064] 00000080:00200000:1.0:1789760788.942894:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.942900:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.942992:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73994240 i_blocks 144544 00000020:00200000:1.0:1789760788.942998:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.943010:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.943045:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73994240, count: 4096 00000080:00200000:1.0:1789760788.943053:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73994240, bytes: 4096 00000080:00200000:1.0:1789760788.943065:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.943073:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.943075:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73994240, 73998336) 0 0 00000080:00200000:1.0:1789760788.943080:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.943095:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18065, 18065] 00020000:00200000:1.0:1789760788.943103:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73994240 chunk: [73994240, 73998336) 73998336, 4096 00020000:00200000:1.0:1789760788.943108:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.943126:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18065, 18065] 00000008:00010000:1.0:1789760788.943142:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.943148:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.943160:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73994240, 73998336) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.943174:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73994240=0x4691000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.943239:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.943242:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 73998336 00000020:00010000:1.0:1789760788.943250:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.943254:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.943257:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.943261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.943264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.943271:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.943277:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.943296:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760788.943796:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 73994240<>73998336 00000100:00100000:0.0:1789760788.944072:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760788.944100:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565543168:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.954228:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985373 gen 3 00000008:00010000:0.0:1789760788.954282:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760788.954290:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.954300:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.954324:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760788.954354:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565543168:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760788.955124:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18065, 18065] 00000080:00200000:1.0:1789760788.955144:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760788.955150:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760788.955166:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 73998336 i_blocks 144552 00000020:00200000:1.0:1789760788.955170:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760788.955181:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760788.955216:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 73998336, count: 4096 00000080:00200000:1.0:1789760788.955224:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 73998336, bytes: 4096 00000080:00200000:1.0:1789760788.955631:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760788.955641:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760788.955661:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [73998336, 74002432) 0 0 00000080:00200000:1.0:1789760788.955667:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760788.955685:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18066, 18066] 00020000:00200000:1.0:1789760788.955694:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 73998336 chunk: [73998336, 74002432) 74002432, 4096 00020000:00200000:1.0:1789760788.955698:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760788.955719:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18066, 18066] 00000008:00010000:1.0:1789760788.955737:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760788.955742:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760788.955755:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [73998336, 74002432) append 0 append lockpos 0 00000080:00200000:1.0:1789760788.955769:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=73998336=0x4692000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760788.957396:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760788.957407:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74002432 00000020:00010000:1.0:1789760788.957424:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760788.957438:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.957442:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760788.957448:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760788.957451:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760788.957464:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760788.957476:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760788.962731:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760788.963425:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 73998336<>74002432 00000100:00100000:1.0:1789760788.963503:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760788.963529:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565543296:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760788.976197:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985374 gen 3 00000008:00010000:0.0:1789760788.976248:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760788.976258:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760788.976269:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760788.976297:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760788.976334:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565543296:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.976733:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18066, 18066] 00000080:00200000:2.0:1789760788.976764:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.976771:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.976786:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74002432 i_blocks 144560 00000020:00200000:2.0:1789760788.976791:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.976801:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.976838:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74002432, count: 4096 00000080:00200000:2.0:1789760788.976847:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74002432, bytes: 4096 00000080:00200000:2.0:1789760788.976859:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.976868:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.976870:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74002432, 74006528) 0 0 00000080:00200000:2.0:1789760788.976875:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.976892:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18067, 18067] 00020000:00200000:2.0:1789760788.976900:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74002432 chunk: [74002432, 74006528) 74006528, 4096 00020000:00200000:2.0:1789760788.976905:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.976927:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18067, 18067] 00000008:00010000:2.0:1789760788.978923:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.979008:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.979028:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74002432, 74006528) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.979046:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74002432=0x4693000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.979125:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.979129:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74006528 00000020:00010000:2.0:1789760788.979138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.979142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.979145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.979149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.979151:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.979160:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.979169:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.979190:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760788.979328:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 74002432<>74006528 00000100:00100000:3.0:1789760788.979424:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565543424:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760788.994847:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985375 gen 3 00000008:00010000:3.0:1789760788.994900:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760788.994908:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760788.994919:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760788.994945:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760788.994975:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565543424:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760788.995179:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18067, 18067] 00000080:00200000:2.0:1789760788.995205:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760788.995211:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760788.995228:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74006528 i_blocks 144568 00000020:00200000:2.0:1789760788.995231:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760788.995242:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760788.995276:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74006528, count: 4096 00000080:00200000:2.0:1789760788.995283:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74006528, bytes: 4096 00000080:00200000:2.0:1789760788.995293:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760788.995302:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760788.995304:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74006528, 74010624) 0 0 00000080:00200000:2.0:1789760788.995308:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760788.995326:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18068, 18068] 00020000:00200000:2.0:1789760788.995335:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74006528 chunk: [74006528, 74010624) 74010624, 4096 00020000:00200000:2.0:1789760788.995339:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760788.995361:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18068, 18068] 00000008:00010000:2.0:1789760788.996687:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760788.996698:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760788.996715:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74006528, 74010624) append 0 append lockpos 0 00000080:00200000:2.0:1789760788.996733:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74006528=0x4694000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760788.996807:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760788.996811:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74010624 00000020:00010000:2.0:1789760788.996820:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760788.996824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.996827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760788.996830:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760788.996833:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760788.996844:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760788.996852:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760788.996879:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760788.997014:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 74006528<>74010624 00000100:00100000:3.0:1789760788.998051:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565543552:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.014902:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985376 gen 3 00000008:00010000:2.0:1789760789.015111:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760789.015120:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.015132:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.015156:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760789.015185:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565543552:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.016615:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18068, 18068] 00000080:00200000:0.0:1789760789.016639:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.016685:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.016704:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74010624 i_blocks 144576 00000020:00200000:0.0:1789760789.016709:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.016724:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.016769:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74010624, count: 4096 00000080:00200000:0.0:1789760789.016778:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74010624, bytes: 4096 00000080:00200000:0.0:1789760789.016795:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.016804:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.016806:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74010624, 74014720) 0 0 00000080:00200000:0.0:1789760789.016811:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.016829:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18069, 18069] 00020000:00200000:0.0:1789760789.016838:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74010624 chunk: [74010624, 74014720) 74014720, 4096 00020000:00200000:0.0:1789760789.016842:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.016865:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18069, 18069] 00000008:00010000:0.0:1789760789.016885:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.016892:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.016906:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74010624, 74014720) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.016926:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74010624=0x4695000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.016986:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.016989:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74014720 00000020:00010000:0.0:1789760789.016997:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.017012:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.017016:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.017020:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.017024:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.017032:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.017040:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.018194:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:2.0:1789760789.018337:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 74010624<>74014720 00000100:00100000:3.0:1789760789.018460:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565543680:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760789.028077:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760789.037277:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985377 gen 3 00000008:00010000:3.0:1789760789.037326:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:3.0:1789760789.037335:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.037347:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.037408:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:3.0:1789760789.037450:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565543680:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.037529:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18069, 18069] 00000080:00200000:0.0:1789760789.037545:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.037550:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.037575:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74014720 i_blocks 144584 00000020:00200000:0.0:1789760789.037579:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.037591:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.037626:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74014720, count: 4096 00000080:00200000:0.0:1789760789.037634:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74014720, bytes: 4096 00000080:00200000:0.0:1789760789.037774:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.037785:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.037787:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74014720, 74018816) 0 0 00000080:00200000:0.0:1789760789.037792:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.037809:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18070, 18070] 00020000:00200000:0.0:1789760789.037818:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74014720 chunk: [74014720, 74018816) 74018816, 4096 00020000:00200000:0.0:1789760789.037823:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.037845:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18070, 18070] 00000008:00010000:0.0:1789760789.037864:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.037871:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.037882:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74014720, 74018816) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.037896:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74014720=0x4696000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.037955:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.037958:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74018816 00000020:00010000:0.0:1789760789.037966:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.037970:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.037974:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.037977:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.037980:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.037988:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.037996:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.038014:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760789.038142:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 74014720<>74018816 00000100:00100000:1.0:1789760789.038348:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565543808:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760789.054025:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985378 gen 3 00000008:00010000:1.0:1789760789.054075:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760789.054083:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760789.054095:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760789.054214:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760789.054248:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565543808:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.055280:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18070, 18070] 00000080:00200000:0.0:1789760789.055299:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.055305:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.055322:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74018816 i_blocks 144592 00000020:00200000:0.0:1789760789.055326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.055340:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.055374:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74018816, count: 4096 00000080:00200000:0.0:1789760789.055381:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74018816, bytes: 4096 00000080:00200000:0.0:1789760789.055392:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.055401:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.055403:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74018816, 74022912) 0 0 00000080:00200000:0.0:1789760789.055408:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.055423:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18071, 18071] 00020000:00200000:0.0:1789760789.057631:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74018816 chunk: [74018816, 74022912) 74022912, 4096 00020000:00200000:0.0:1789760789.057659:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.057698:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18071, 18071] 00000008:00010000:0.0:1789760789.057727:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.057735:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.057749:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74018816, 74022912) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.057768:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74018816=0x4697000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.057862:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.057867:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74022912 00000020:00010000:0.0:1789760789.057875:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.057879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.057882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.057885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.057889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.057902:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.057912:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.057946:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760789.058105:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 74018816<>74022912 00000100:00100000:1.0:1789760789.058188:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760789.058218:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565543936:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760789.088077:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985379 gen 3 00000008:00010000:1.0:1789760789.088126:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760789.088135:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760789.088145:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760789.088210:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760789.088236:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565543936:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.088386:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18071, 18071] 00000080:00200000:0.0:1789760789.088402:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.088408:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.088552:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74022912 i_blocks 144600 00000020:00200000:1.0:1789760789.088557:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.088569:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.088604:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74022912, count: 4096 00000080:00200000:1.0:1789760789.088612:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74022912, bytes: 4096 00000080:00200000:1.0:1789760789.088907:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.088918:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.088920:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74022912, 74027008) 0 0 00000080:00200000:1.0:1789760789.088925:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.088940:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18072, 18072] 00020000:00200000:1.0:1789760789.088948:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74022912 chunk: [74022912, 74027008) 74027008, 4096 00020000:00200000:1.0:1789760789.088953:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.088973:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18072, 18072] 00000008:00010000:1.0:1789760789.088990:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.088995:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.089006:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74022912, 74027008) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.089020:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74022912=0x4698000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.089073:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.089076:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74027008 00000020:00010000:1.0:1789760789.089084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.089087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.089090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.089094:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.089097:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.089103:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.089109:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.089125:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760789.089240:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74022912<>74027008 00000100:00100000:0.0:1789760789.089394:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565544064:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760789.102030:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985380 gen 3 00000008:00010000:1.0:1789760789.102082:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760789.102091:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760789.102102:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760789.102129:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760789.102157:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565544064:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.102287:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18072, 18072] 00000080:00200000:2.0:1789760789.102315:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.102321:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.102337:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74027008 i_blocks 144608 00000020:00200000:2.0:1789760789.102340:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.102349:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.102559:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74027008, count: 4096 00000080:00200000:2.0:1789760789.102569:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74027008, bytes: 4096 00000080:00200000:2.0:1789760789.102580:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.102589:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.102592:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74027008, 74031104) 0 0 00000080:00200000:2.0:1789760789.102596:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.102614:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18073, 18073] 00020000:00200000:2.0:1789760789.102625:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74027008 chunk: [74027008, 74031104) 74031104, 4096 00020000:00200000:2.0:1789760789.102631:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.102688:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18073, 18073] 00000008:00010000:2.0:1789760789.102707:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.102713:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.102725:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74027008, 74031104) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.102740:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74027008=0x4699000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.102805:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.102808:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74031104 00000020:00010000:2.0:1789760789.102817:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.102820:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.102824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.102827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.102831:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.102839:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.102846:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.102864:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.102993:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 74027008<>74031104 00000100:00100000:3.0:1789760789.103081:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565544192:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.117225:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985381 gen 3 00000008:00010000:2.0:1789760789.117277:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760789.117285:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.117296:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.117984:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760789.118021:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565544192:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.122178:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18073, 18073] 00000080:00200000:1.0:1789760789.122208:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.122216:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.122235:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74031104 i_blocks 144616 00000020:00200000:1.0:1789760789.122241:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.122259:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.122298:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74031104, count: 4096 00000080:00200000:1.0:1789760789.122441:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74031104, bytes: 4096 00000080:00200000:1.0:1789760789.122458:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.122469:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.122471:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74031104, 74035200) 0 0 00000080:00200000:1.0:1789760789.122476:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.122495:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18074, 18074] 00020000:00200000:1.0:1789760789.122505:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74031104 chunk: [74031104, 74035200) 74035200, 4096 00020000:00200000:1.0:1789760789.122509:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.122533:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18074, 18074] 00000008:00010000:1.0:1789760789.122554:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.122561:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.122573:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74031104, 74035200) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.122589:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74031104=0x469a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.122685:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.122689:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74035200 00000020:00010000:1.0:1789760789.122697:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.122701:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.122703:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.122707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.122710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.122719:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.122726:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.122757:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760789.123252:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74031104<>74035200 00000100:00100000:0.0:1789760789.123475:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760789.130107:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565544320:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760789.140619:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985382 gen 3 00000008:00010000:1.0:1789760789.140719:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760789.140728:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760789.140739:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760789.140753:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760789.140798:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565544320:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.141067:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18074, 18074] 00000080:00200000:0.0:1789760789.141083:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.141089:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.141104:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74035200 i_blocks 144624 00000020:00200000:0.0:1789760789.141108:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.141118:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.141153:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74035200, count: 4096 00000080:00200000:0.0:1789760789.141161:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74035200, bytes: 4096 00000080:00200000:0.0:1789760789.141173:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.141181:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.141184:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74035200, 74039296) 0 0 00000080:00200000:0.0:1789760789.141188:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.141206:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18075, 18075] 00020000:00200000:0.0:1789760789.141215:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74035200 chunk: [74035200, 74039296) 74039296, 4096 00020000:00200000:0.0:1789760789.141220:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.141240:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18075, 18075] 00000008:00010000:0.0:1789760789.141258:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.141264:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.141276:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74035200, 74039296) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.141290:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74035200=0x469b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.141353:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.141357:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74039296 00000020:00010000:0.0:1789760789.141366:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.141370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.141454:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.141460:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.141463:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.141472:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.141479:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.141510:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760789.141637:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 74035200<>74039296 00000100:00100000:1.0:1789760789.141914:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760789.141935:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565544448:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760789.156868:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985383 gen 3 00000008:00010000:1.0:1789760789.156923:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760789.156932:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760789.156944:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760789.156962:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760789.156996:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565544448:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.157159:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18075, 18075] 00000080:00200000:0.0:1789760789.157177:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.157183:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.157199:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74039296 i_blocks 144632 00000020:00200000:0.0:1789760789.157203:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.157214:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.157249:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74039296, count: 4096 00000080:00200000:0.0:1789760789.157256:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74039296, bytes: 4096 00000080:00200000:0.0:1789760789.157271:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.157280:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.157283:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74039296, 74043392) 0 0 00000080:00200000:0.0:1789760789.157288:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.157306:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18076, 18076] 00020000:00200000:0.0:1789760789.157316:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74039296 chunk: [74039296, 74043392) 74043392, 4096 00020000:00200000:0.0:1789760789.157321:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.157342:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18076, 18076] 00000008:00010000:0.0:1789760789.157362:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.157380:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.157393:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74039296, 74043392) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.157408:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74039296=0x469c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.157480:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.157484:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74043392 00000020:00010000:0.0:1789760789.157492:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.157496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.157499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.157502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.157505:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.157513:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.157519:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.157536:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dcc0 00000008:00100000:0.0:1789760789.157698:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74039296<>74043392 00000100:00100000:1.0:1789760789.157867:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565544576:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760789.171264:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985384 gen 3 00000008:00010000:1.0:1789760789.171318:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760789.171327:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760789.171339:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760789.171361:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760789.171390:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565544576:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.172474:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18076, 18076] 00000080:00200000:0.0:1789760789.172493:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.172500:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.172518:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74043392 i_blocks 144640 00000020:00200000:0.0:1789760789.172523:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.172535:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.172569:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74043392, count: 4096 00000080:00200000:0.0:1789760789.172576:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74043392, bytes: 4096 00000080:00200000:0.0:1789760789.172588:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.172597:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.172600:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74043392, 74047488) 0 0 00000080:00200000:0.0:1789760789.172604:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.172620:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18077, 18077] 00020000:00200000:0.0:1789760789.172629:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74043392 chunk: [74043392, 74047488) 74047488, 4096 00020000:00200000:0.0:1789760789.172633:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.173455:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18077, 18077] 00000008:00010000:2.0:1789760789.173484:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.173491:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.173505:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74043392, 74047488) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.173520:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74043392=0x469d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.173590:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.173593:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74047488 00000020:00010000:2.0:1789760789.173602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.173606:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.173609:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.173612:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.173615:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.173626:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.173634:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.173887:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.174957:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 74043392<>74047488 00000100:00100000:3.0:1789760789.175066:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565544704:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.188347:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985385 gen 3 00000008:00010000:3.0:1789760789.188399:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760789.188408:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.188420:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.188457:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760789.188491:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565544704:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.188609:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18077, 18077] 00000080:00200000:2.0:1789760789.188636:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.188673:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.188689:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74047488 i_blocks 144648 00000020:00200000:2.0:1789760789.188693:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.188705:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.188740:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74047488, count: 4096 00000080:00200000:2.0:1789760789.188748:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74047488, bytes: 4096 00000080:00200000:2.0:1789760789.188758:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.188767:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.188770:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74047488, 74051584) 0 0 00000080:00200000:2.0:1789760789.188774:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.188791:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18078, 18078] 00020000:00200000:2.0:1789760789.188801:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74047488 chunk: [74047488, 74051584) 74051584, 4096 00020000:00200000:2.0:1789760789.188807:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.188827:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18078, 18078] 00000008:00010000:2.0:1789760789.188846:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.188852:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.188864:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74047488, 74051584) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.188879:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74047488=0x469e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.188935:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.188938:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74051584 00000020:00010000:2.0:1789760789.188947:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.188950:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.188954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.188958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.188960:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.188968:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.188975:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.188991:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760789.189110:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 74047488<>74051584 00000100:00100000:3.0:1789760789.189291:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760789.189310:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565544832:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.199555:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985386 gen 3 00000008:00010000:3.0:1789760789.199608:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760789.199616:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.199627:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.199669:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760789.199701:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565544832:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.199725:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18078, 18078] 00000080:00200000:2.0:1789760789.199746:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.199753:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.199771:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74051584 i_blocks 144656 00000020:00200000:2.0:1789760789.199775:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.199785:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.199822:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74051584, count: 4096 00000080:00200000:2.0:1789760789.199829:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74051584, bytes: 4096 00000080:00200000:2.0:1789760789.199839:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.199849:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.199851:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74051584, 74055680) 0 0 00000080:00200000:2.0:1789760789.199856:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.199874:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18079, 18079] 00020000:00200000:2.0:1789760789.199884:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74051584 chunk: [74051584, 74055680) 74055680, 4096 00020000:00200000:2.0:1789760789.199890:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.199911:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18079, 18079] 00000008:00010000:2.0:1789760789.199932:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.199939:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.199952:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74051584, 74055680) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.199966:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74051584=0x469f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.200029:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.200032:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74055680 00000020:00010000:2.0:1789760789.200041:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.200044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.200047:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.200051:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.200054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.200061:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.200067:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.200084:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760789.200211:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 74051584<>74055680 00000100:00100000:3.0:1789760789.200339:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565544960:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.219485:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985387 gen 3 00000008:00010000:3.0:1789760789.221238:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760789.221253:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.221270:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.221385:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760789.221419:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565544960:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.221751:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18079, 18079] 00000080:00200000:2.0:1789760789.221770:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.221775:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.221792:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74055680 i_blocks 144664 00000020:00200000:2.0:1789760789.221796:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.221806:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.221842:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74055680, count: 4096 00000080:00200000:2.0:1789760789.221849:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74055680, bytes: 4096 00000080:00200000:2.0:1789760789.221864:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.221873:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.221875:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74055680, 74059776) 0 0 00000080:00200000:2.0:1789760789.221880:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.221896:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18080, 18080] 00020000:00200000:2.0:1789760789.221905:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74055680 chunk: [74055680, 74059776) 74059776, 4096 00020000:00200000:2.0:1789760789.222069:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.222156:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18080, 18080] 00000008:00010000:2.0:1789760789.222174:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.222181:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.222193:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74055680, 74059776) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.222208:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74055680=0x46a0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.222274:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.222277:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74059776 00000020:00010000:2.0:1789760789.222285:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.222289:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.222292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.222296:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.222298:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.222306:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.222313:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.222333:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.222476:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74055680<>74059776 00000100:00100000:3.0:1789760789.222575:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565545088:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.234792:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985388 gen 3 00000008:00010000:2.0:1789760789.234842:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760789.234850:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.234861:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.234950:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760789.234995:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565545088:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.237188:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18080, 18080] 00000080:00200000:0.0:1789760789.237216:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.237223:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.237242:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74059776 i_blocks 144672 00000020:00200000:0.0:1789760789.237248:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.237266:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.237305:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74059776, count: 4096 00000080:00200000:0.0:1789760789.237314:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74059776, bytes: 4096 00000080:00200000:0.0:1789760789.237329:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.237339:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.237341:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74059776, 74063872) 0 0 00000080:00200000:0.0:1789760789.237347:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.237632:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18081, 18081] 00020000:00200000:0.0:1789760789.237665:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74059776 chunk: [74059776, 74063872) 74063872, 4096 00020000:00200000:0.0:1789760789.237671:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.237696:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18081, 18081] 00000008:00010000:0.0:1789760789.237716:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.237722:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.237737:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74059776, 74063872) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.237753:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74059776=0x46a1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.237906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.237909:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74063872 00000020:00010000:0.0:1789760789.237918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.237922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.237925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.237929:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.237932:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.237941:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.237949:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.237984:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760789.238572:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74059776<>74063872 00000100:00100000:1.0:1789760789.238771:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760789.238804:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565545216:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.253297:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985389 gen 3 00000008:00010000:0.0:1789760789.253349:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760789.253358:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.253443:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.253587:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760789.253619:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565545216:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.253868:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18081, 18081] 00000080:00200000:2.0:1789760789.253885:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.253891:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.253907:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74063872 i_blocks 144680 00000020:00200000:2.0:1789760789.253911:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.253923:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.254035:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74063872, count: 4096 00000080:00200000:2.0:1789760789.254044:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74063872, bytes: 4096 00000080:00200000:2.0:1789760789.254055:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.254138:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.254141:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74063872, 74067968) 0 0 00000080:00200000:2.0:1789760789.254146:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.254162:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18082, 18082] 00020000:00200000:2.0:1789760789.254171:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74063872 chunk: [74063872, 74067968) 74067968, 4096 00020000:00200000:2.0:1789760789.254175:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.254196:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18082, 18082] 00000008:00010000:2.0:1789760789.254213:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.254219:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.254230:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74063872, 74067968) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.254244:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74063872=0x46a2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.255086:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.255092:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74067968 00000020:00010000:2.0:1789760789.255104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.255110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.255113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.255117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.255120:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.255129:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.255138:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.255167:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.255300:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74063872<>74067968 00000100:00100000:2.0:1789760789.255359:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760789.255393:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565545344:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.262079:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985390 gen 3 00000008:00010000:2.0:1789760789.262132:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760789.262142:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.262155:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.262181:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760789.262211:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565545344:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.262338:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18082, 18082] 00000080:00200000:0.0:1789760789.262357:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.262364:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.262391:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74067968 i_blocks 144688 00000020:00200000:0.0:1789760789.262395:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.262406:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.262452:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74067968, count: 4096 00000080:00200000:0.0:1789760789.262461:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74067968, bytes: 4096 00000080:00200000:0.0:1789760789.262471:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.262480:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.262483:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74067968, 74072064) 0 0 00000080:00200000:0.0:1789760789.262487:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.262506:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18083, 18083] 00020000:00200000:0.0:1789760789.262515:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74067968 chunk: [74067968, 74072064) 74072064, 4096 00020000:00200000:0.0:1789760789.262520:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.262541:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18083, 18083] 00000008:00010000:0.0:1789760789.262560:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.262566:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.262578:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74067968, 74072064) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.262595:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74067968=0x46a3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.262695:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.262699:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74072064 00000020:00010000:0.0:1789760789.262707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.262710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.262713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.262717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.262720:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.262728:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.262736:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.262753:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760789.262875:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74067968<>74072064 00000100:00100000:1.0:1789760789.263119:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565545472:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.278123:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985391 gen 3 00000008:00010000:0.0:1789760789.278173:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760789.278182:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.278194:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.278220:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760789.278251:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565545472:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.279120:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18083, 18083] 00000080:00200000:2.0:1789760789.279142:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.279149:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.279166:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74072064 i_blocks 144696 00000020:00200000:2.0:1789760789.279170:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.279184:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.279217:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74072064, count: 4096 00000080:00200000:2.0:1789760789.279223:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74072064, bytes: 4096 00000080:00200000:2.0:1789760789.279234:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.279241:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.279244:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74072064, 74076160) 0 0 00000080:00200000:2.0:1789760789.279248:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.279264:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18084, 18084] 00020000:00200000:2.0:1789760789.279271:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74072064 chunk: [74072064, 74076160) 74076160, 4096 00020000:00200000:2.0:1789760789.279275:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.279293:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18084, 18084] 00000008:00010000:2.0:1789760789.279309:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.279314:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.279324:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74072064, 74076160) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.279337:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74072064=0x46a4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.280177:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.280185:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74076160 00000020:00010000:2.0:1789760789.280199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.280202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.280206:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.280209:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.280212:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.280222:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.280229:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.280257:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.280390:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74072064<>74076160 00000100:00100000:3.0:1789760789.280551:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760789.280577:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565545600:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.293606:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985392 gen 3 00000008:00010000:3.0:1789760789.293862:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760789.293872:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.293884:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.296464:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760789.296517:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565545600:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.297320:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18084, 18084] 00000080:00200000:2.0:1789760789.297341:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.297347:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.297375:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74076160 i_blocks 144704 00000020:00200000:2.0:1789760789.297381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.297394:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.297439:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74076160, count: 4096 00000080:00200000:2.0:1789760789.297446:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74076160, bytes: 4096 00000080:00200000:2.0:1789760789.297459:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.297468:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.297470:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74076160, 74080256) 0 0 00000080:00200000:2.0:1789760789.297474:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.297490:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18085, 18085] 00020000:00200000:2.0:1789760789.297500:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74076160 chunk: [74076160, 74080256) 74080256, 4096 00020000:00200000:2.0:1789760789.297505:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.297526:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18085, 18085] 00000008:00010000:2.0:1789760789.297543:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.297549:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.297562:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74076160, 74080256) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.297576:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74076160=0x46a5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.297687:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.297690:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74080256 00000020:00010000:2.0:1789760789.297699:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.297703:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.297706:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.297710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.297713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.297722:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.297729:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.297747:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.297876:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 74076160<>74080256 00000100:00100000:3.0:1789760789.298832:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565545728:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.312206:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985393 gen 3 00000008:00010000:3.0:1789760789.312258:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760789.312267:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.312277:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.312302:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760789.312334:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565545728:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.312626:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18085, 18085] 00000080:00200000:2.0:1789760789.312681:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.312688:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.312705:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74080256 i_blocks 144712 00000020:00200000:2.0:1789760789.312709:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.312720:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.312754:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74080256, count: 4096 00000080:00200000:2.0:1789760789.312762:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74080256, bytes: 4096 00000080:00200000:2.0:1789760789.312892:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.312902:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.312905:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74080256, 74084352) 0 0 00000080:00200000:2.0:1789760789.312910:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.312928:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18086, 18086] 00020000:00200000:2.0:1789760789.312938:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74080256 chunk: [74080256, 74084352) 74084352, 4096 00020000:00200000:2.0:1789760789.312943:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.312970:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18086, 18086] 00000008:00010000:2.0:1789760789.312989:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.312995:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.313008:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74080256, 74084352) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.313023:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74080256=0x46a6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.313085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.313088:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74084352 00000020:00010000:2.0:1789760789.313096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.313100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.313103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.313106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.313108:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.313117:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.313124:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.313142:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760789.313269:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 74080256<>74084352 00000100:00100000:3.0:1789760789.313396:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565545856:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.336320:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985394 gen 3 00000008:00010000:3.0:1789760789.336385:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760789.336394:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.336408:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.336437:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760789.336467:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565545856:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.338045:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18086, 18086] 00000080:00200000:2.0:1789760789.338073:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.338079:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.338100:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74084352 i_blocks 144720 00000020:00200000:2.0:1789760789.338106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.338122:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.338162:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74084352, count: 4096 00000080:00200000:2.0:1789760789.338170:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74084352, bytes: 4096 00000080:00200000:2.0:1789760789.338186:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.338195:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.338198:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74084352, 74088448) 0 0 00000080:00200000:2.0:1789760789.338202:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.338221:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18087, 18087] 00020000:00200000:2.0:1789760789.338231:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74084352 chunk: [74084352, 74088448) 74088448, 4096 00020000:00200000:2.0:1789760789.338235:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.338259:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18087, 18087] 00000008:00010000:2.0:1789760789.338278:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.338284:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.338297:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74084352, 74088448) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.338312:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74084352=0x46a7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.338984:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.338990:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74088448 00000020:00010000:2.0:1789760789.339001:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.339005:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.339009:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.339013:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.339016:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.339027:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.339035:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.339055:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189c00 00000008:00100000:2.0:1789760789.339195:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 74084352<>74088448 00000100:00100000:3.0:1789760789.339392:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565545984:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.352129:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985395 gen 3 00000008:00010000:3.0:1789760789.352181:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189c00 00000020:00200000:3.0:1789760789.352189:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.352201:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.352226:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189c00 00000100:00100000:3.0:1789760789.352260:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565545984:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.352390:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18087, 18087] 00000080:00200000:2.0:1789760789.352408:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.352413:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.352439:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74088448 i_blocks 144728 00000020:00200000:2.0:1789760789.352443:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.352452:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.352604:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74088448, count: 4096 00000080:00200000:2.0:1789760789.352613:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74088448, bytes: 4096 00000080:00200000:2.0:1789760789.352623:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.352632:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.352635:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74088448, 74092544) 0 0 00000080:00200000:2.0:1789760789.352640:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.352749:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18088, 18088] 00020000:00200000:2.0:1789760789.352759:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74088448 chunk: [74088448, 74092544) 74092544, 4096 00020000:00200000:2.0:1789760789.352763:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.352857:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18088, 18088] 00000008:00010000:2.0:1789760789.352876:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.352883:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.352897:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74088448, 74092544) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.352911:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74088448=0x46a8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.352972:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.352975:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74092544 00000020:00010000:2.0:1789760789.352984:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.352987:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.352990:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.352994:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.352996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.353005:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.353011:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.353029:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.353151:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 74088448<>74092544 00000100:00100000:3.0:1789760789.353225:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565546112:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.363550:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985396 gen 3 00000008:00010000:2.0:1789760789.363602:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760789.363610:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.363621:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.363681:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760789.363711:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565546112:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.363868:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18088, 18088] 00000080:00200000:0.0:1789760789.363885:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.363892:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.363908:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74092544 i_blocks 144736 00000020:00200000:0.0:1789760789.363912:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.363923:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.363959:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74092544, count: 4096 00000080:00200000:0.0:1789760789.363967:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74092544, bytes: 4096 00000080:00200000:0.0:1789760789.363977:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.363987:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.363990:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74092544, 74096640) 0 0 00000080:00200000:0.0:1789760789.363994:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.364012:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18089, 18089] 00020000:00200000:0.0:1789760789.364021:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74092544 chunk: [74092544, 74096640) 74096640, 4096 00020000:00200000:0.0:1789760789.364026:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.364046:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18089, 18089] 00000008:00010000:0.0:1789760789.364065:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.364070:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.364082:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74092544, 74096640) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.364096:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74092544=0x46a9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.364156:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.364159:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74096640 00000020:00010000:0.0:1789760789.364167:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.364172:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.364175:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.364178:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.364181:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.364188:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.364195:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.364212:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760789.364335:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74092544<>74096640 00000100:00100000:1.0:1789760789.364410:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565546240:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.374057:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985397 gen 3 00000008:00010000:0.0:1789760789.374109:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760789.374117:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.374128:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.374155:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760789.374186:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565546240:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.374351:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18089, 18089] 00000080:00200000:1.0:1789760789.374378:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.374385:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.374401:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74096640 i_blocks 144744 00000020:00200000:1.0:1789760789.374404:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.374416:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.374459:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74096640, count: 4096 00000080:00200000:1.0:1789760789.374467:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74096640, bytes: 4096 00000080:00200000:1.0:1789760789.374479:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.374499:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.374502:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74096640, 74100736) 0 0 00000080:00200000:1.0:1789760789.374506:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.374525:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18090, 18090] 00020000:00200000:1.0:1789760789.374535:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74096640 chunk: [74096640, 74100736) 74100736, 4096 00020000:00200000:1.0:1789760789.374540:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.374562:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18090, 18090] 00000008:00010000:1.0:1789760789.374580:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.374586:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.374598:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74096640, 74100736) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.374613:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74096640=0x46aa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.374707:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.374711:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74100736 00000020:00010000:1.0:1789760789.374719:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.374722:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.374725:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.374728:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.374731:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.374738:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.374745:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.374762:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760789.374901:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74096640<>74100736 00000100:00100000:0.0:1789760789.375832:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565546368:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.386135:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985398 gen 3 00000008:00010000:0.0:1789760789.386189:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760789.386197:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.386209:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.386270:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760789.386302:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565546368:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.386459:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18090, 18090] 00000080:00200000:1.0:1789760789.386476:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.386482:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.386566:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74100736 i_blocks 144752 00000020:00200000:1.0:1789760789.386571:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.386581:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.386617:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74100736, count: 4096 00000080:00200000:1.0:1789760789.386625:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74100736, bytes: 4096 00000080:00200000:1.0:1789760789.386635:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.386678:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.386680:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74100736, 74104832) 0 0 00000080:00200000:1.0:1789760789.386685:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.386702:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18091, 18091] 00020000:00200000:1.0:1789760789.386711:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74100736 chunk: [74100736, 74104832) 74104832, 4096 00020000:00200000:1.0:1789760789.386715:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.386736:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18091, 18091] 00000008:00010000:1.0:1789760789.386754:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.386760:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.386770:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74100736, 74104832) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.386783:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74100736=0x46ab000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.386843:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.386846:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74104832 00000020:00010000:1.0:1789760789.386854:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.386857:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.386861:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.386864:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.386867:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.386875:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.386883:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.386899:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760789.387156:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74100736<>74104832 00000100:00100000:0.0:1789760789.387240:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565546496:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.397257:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985399 gen 3 00000008:00010000:0.0:1789760789.397309:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760789.397318:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.397330:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.397351:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760789.397454:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565546496:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.397621:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18091, 18091] 00000080:00200000:1.0:1789760789.397638:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.397683:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.397700:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74104832 i_blocks 144760 00000020:00200000:1.0:1789760789.397704:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.397713:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.397749:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74104832, count: 4096 00000080:00200000:1.0:1789760789.397757:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74104832, bytes: 4096 00000080:00200000:1.0:1789760789.397766:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.397892:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.397894:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74104832, 74108928) 0 0 00000080:00200000:1.0:1789760789.397899:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.397916:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18092, 18092] 00020000:00200000:1.0:1789760789.397926:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74104832 chunk: [74104832, 74108928) 74108928, 4096 00020000:00200000:1.0:1789760789.397930:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.397951:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18092, 18092] 00000008:00010000:1.0:1789760789.397970:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.397977:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.397988:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74104832, 74108928) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.398003:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74104832=0x46ac000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.398066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.398069:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74108928 00000020:00010000:1.0:1789760789.398078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.398081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.398084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.398087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.398091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.398098:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.398106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.398124:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760789.398253:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 74104832<>74108928 00000100:00100000:0.0:1789760789.398329:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565546624:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.413154:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985400 gen 3 00000008:00010000:0.0:1789760789.413205:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760789.413215:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.413226:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.413292:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760789.413323:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565546624:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.413913:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18092, 18092] 00000080:00200000:1.0:1789760789.413932:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.413938:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.413954:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74108928 i_blocks 144768 00000020:00200000:1.0:1789760789.413958:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.413969:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.414005:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74108928, count: 4096 00000080:00200000:1.0:1789760789.414013:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74108928, bytes: 4096 00000080:00200000:1.0:1789760789.414026:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.414036:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.414039:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74108928, 74113024) 0 0 00000080:00200000:1.0:1789760789.414043:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.414062:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18093, 18093] 00020000:00200000:1.0:1789760789.414071:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74108928 chunk: [74108928, 74113024) 74113024, 4096 00020000:00200000:1.0:1789760789.414076:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.414094:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18093, 18093] 00000008:00010000:1.0:1789760789.414112:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.414119:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.414130:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74108928, 74113024) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.414145:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74108928=0x46ad000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.414208:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.414212:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74113024 00000020:00010000:1.0:1789760789.414220:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.414223:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.414227:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.414230:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.414233:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.414240:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.414249:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.414273:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760789.414473:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 74108928<>74113024 00000100:00100000:0.0:1789760789.414605:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565546752:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.424832:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985401 gen 3 00000008:00010000:0.0:1789760789.424881:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760789.424890:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.424900:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.424919:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760789.424949:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565546752:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.425218:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18093, 18093] 00000080:00200000:1.0:1789760789.425236:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.425242:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.425258:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74113024 i_blocks 144776 00000020:00200000:1.0:1789760789.425262:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.425273:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.425316:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74113024, count: 4096 00000080:00200000:1.0:1789760789.425323:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74113024, bytes: 4096 00000080:00200000:1.0:1789760789.425334:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.425343:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.425345:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74113024, 74117120) 0 0 00000080:00200000:1.0:1789760789.425349:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.425437:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18094, 18094] 00020000:00200000:1.0:1789760789.425448:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74113024 chunk: [74113024, 74117120) 74117120, 4096 00020000:00200000:1.0:1789760789.425453:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.425626:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18094, 18094] 00000008:00010000:1.0:1789760789.425743:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.425750:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.425763:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74113024, 74117120) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.425943:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74113024=0x46ae000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.426074:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.426078:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74117120 00000020:00010000:1.0:1789760789.426087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.426091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.426094:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.426154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.426159:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.426169:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.426177:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.426206:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760789.426345:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 74113024<>74117120 00000100:00100000:0.0:1789760789.426528:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565546880:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.444060:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985402 gen 3 00000008:00010000:0.0:1789760789.444177:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760789.444187:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.444199:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.444268:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760789.444301:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565546880:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.444461:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18094, 18094] 00000080:00200000:1.0:1789760789.444610:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.444617:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.444635:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74117120 i_blocks 144784 00000020:00200000:1.0:1789760789.444639:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.444721:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.444757:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74117120, count: 4096 00000080:00200000:1.0:1789760789.444765:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74117120, bytes: 4096 00000080:00200000:1.0:1789760789.444850:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.444860:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.444863:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74117120, 74121216) 0 0 00000080:00200000:1.0:1789760789.444867:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.444887:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18095, 18095] 00020000:00200000:1.0:1789760789.444896:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74117120 chunk: [74117120, 74121216) 74121216, 4096 00020000:00200000:1.0:1789760789.444901:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.445047:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18095, 18095] 00000008:00010000:1.0:1789760789.445069:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.445075:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.445158:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74117120, 74121216) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.445175:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74117120=0x46af000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.445243:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.445247:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74121216 00000020:00010000:1.0:1789760789.445255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.445258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.445262:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.445264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.445267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.445275:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.445282:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.445306:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760789.445637:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 74117120<>74121216 00000100:00100000:0.0:1789760789.445747:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565547008:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.461298:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985403 gen 3 00000008:00010000:0.0:1789760789.461348:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760789.461356:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.461380:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.461400:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760789.461442:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565547008:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.461596:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18095, 18095] 00000080:00200000:1.0:1789760789.461614:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.461620:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.461637:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74121216 i_blocks 144792 00000020:00200000:1.0:1789760789.461641:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.461696:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.461729:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74121216, count: 4096 00000080:00200000:1.0:1789760789.461736:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74121216, bytes: 4096 00000080:00200000:1.0:1789760789.461746:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.461754:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.461757:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74121216, 74125312) 0 0 00000080:00200000:1.0:1789760789.461762:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.461780:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18096, 18096] 00020000:00200000:1.0:1789760789.461789:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74121216 chunk: [74121216, 74125312) 74125312, 4096 00020000:00200000:1.0:1789760789.461794:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.461813:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18096, 18096] 00000008:00010000:1.0:1789760789.461830:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.461836:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.461846:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74121216, 74125312) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.461861:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74121216=0x46b0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.461924:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.461927:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74125312 00000020:00010000:1.0:1789760789.461936:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.461939:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.461943:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.461946:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.461949:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.461955:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.461964:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.461982:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760789.462123:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 74121216<>74125312 00000100:00100000:0.0:1789760789.462288:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565547136:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.476207:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985404 gen 3 00000008:00010000:0.0:1789760789.476257:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760789.476266:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.476279:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.476352:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760789.476382:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565547136:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.478824:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18096, 18096] 00000080:00200000:1.0:1789760789.478851:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.478858:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.478998:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74125312 i_blocks 144800 00000020:00200000:2.0:1789760789.479005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.479023:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.479069:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74125312, count: 4096 00000080:00200000:2.0:1789760789.479247:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74125312, bytes: 4096 00000080:00200000:2.0:1789760789.479267:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.479277:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.479280:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74125312, 74129408) 0 0 00000080:00200000:2.0:1789760789.479285:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.479302:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18097, 18097] 00020000:00200000:2.0:1789760789.479312:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74125312 chunk: [74125312, 74129408) 74129408, 4096 00020000:00200000:2.0:1789760789.479316:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.479341:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18097, 18097] 00000008:00010000:2.0:1789760789.479361:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.479368:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.479381:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74125312, 74129408) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.479396:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74125312=0x46b1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.479479:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.479482:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74129408 00000020:00010000:2.0:1789760789.479490:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.479494:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.479498:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.479502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.479504:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.479513:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.479520:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.479586:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.479757:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 74125312<>74129408 00000100:00100000:2.0:1789760789.479816:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760789.479841:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565547264:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.497753:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985405 gen 3 00000008:00010000:2.0:1789760789.497805:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760789.497813:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.497824:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.497855:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760789.497889:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565547264:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.498028:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18097, 18097] 00000080:00200000:0.0:1789760789.498044:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.498050:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.498065:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74129408 i_blocks 144808 00000020:00200000:0.0:1789760789.498070:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.498092:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.498127:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74129408, count: 4096 00000080:00200000:0.0:1789760789.498135:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74129408, bytes: 4096 00000080:00200000:0.0:1789760789.498149:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.498158:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.498161:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74129408, 74133504) 0 0 00000080:00200000:0.0:1789760789.498165:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.498182:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18098, 18098] 00020000:00200000:0.0:1789760789.498191:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74129408 chunk: [74129408, 74133504) 74133504, 4096 00020000:00200000:0.0:1789760789.498196:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.498216:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18098, 18098] 00000008:00010000:0.0:1789760789.498233:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.498241:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.498252:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74129408, 74133504) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.498266:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74129408=0x46b2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.498325:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.498328:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74133504 00000020:00010000:0.0:1789760789.498336:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.498340:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.498343:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.498347:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.498350:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.498358:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.498365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.498383:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760789.498511:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74129408<>74133504 00000100:00100000:0.0:1789760789.498562:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760789.498578:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565547392:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.513818:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985406 gen 3 00000008:00010000:0.0:1789760789.513870:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760789.513878:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.513890:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.513918:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760789.513948:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565547392:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.514200:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18098, 18098] 00000080:00200000:2.0:1789760789.514217:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.514223:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.514239:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74133504 i_blocks 144816 00000020:00200000:2.0:1789760789.514242:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.514254:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.514289:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74133504, count: 4096 00000080:00200000:2.0:1789760789.514297:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74133504, bytes: 4096 00000080:00200000:2.0:1789760789.514308:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.514317:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.514320:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74133504, 74137600) 0 0 00000080:00200000:2.0:1789760789.514325:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.514343:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18099, 18099] 00020000:00200000:2.0:1789760789.514352:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74133504 chunk: [74133504, 74137600) 74137600, 4096 00020000:00200000:2.0:1789760789.514358:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.514388:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18099, 18099] 00000008:00010000:2.0:1789760789.514408:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.514414:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.514435:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74133504, 74137600) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.514450:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74133504=0x46b3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.514508:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.514511:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74137600 00000020:00010000:2.0:1789760789.514520:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.514523:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.514527:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.514531:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.514534:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.514542:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.514548:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.514565:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.514735:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 74133504<>74137600 00000100:00100000:3.0:1789760789.514937:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760789.514974:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565547520:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.531632:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985407 gen 3 00000008:00010000:3.0:1789760789.531733:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760789.531743:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.531754:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.531838:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000080:00200000:2.0:1789760789.531841:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18099, 18099] 00000080:00200000:2.0:1789760789.531861:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.531866:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000100:00100000:3.0:1789760789.531871:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565547520:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.531882:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74137600 i_blocks 144824 00000020:00200000:2.0:1789760789.531885:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.531896:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.531931:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74137600, count: 4096 00000080:00200000:2.0:1789760789.531939:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74137600, bytes: 4096 00000080:00200000:2.0:1789760789.531951:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.531965:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.531968:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74137600, 74141696) 0 0 00000080:00200000:2.0:1789760789.531972:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.531989:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18100, 18100] 00020000:00200000:2.0:1789760789.531998:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74137600 chunk: [74137600, 74141696) 74141696, 4096 00020000:00200000:2.0:1789760789.532004:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.532026:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18100, 18100] 00000008:00010000:2.0:1789760789.532045:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.532051:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.532062:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74137600, 74141696) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.532077:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74137600=0x46b4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.532128:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.532130:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74141696 00000020:00010000:2.0:1789760789.532137:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.532140:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.532142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.532145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.532147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.532153:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.532161:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.532175:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.532304:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 74137600<>74141696 00000100:00100000:3.0:1789760789.532526:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565547648:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.549885:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985408 gen 3 00000008:00010000:2.0:1789760789.550083:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:2.0:1789760789.550091:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.550102:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.550120:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:2.0:1789760789.550150:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565547648:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.550152:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18100, 18100] 00000080:00200000:0.0:1789760789.550170:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.550176:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.550190:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74141696 i_blocks 144832 00000020:00200000:0.0:1789760789.550194:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.550204:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.550240:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74141696, count: 4096 00000080:00200000:0.0:1789760789.550248:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74141696, bytes: 4096 00000080:00200000:0.0:1789760789.550259:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.550267:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.550269:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74141696, 74145792) 0 0 00000080:00200000:0.0:1789760789.550273:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.550288:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18101, 18101] 00020000:00200000:0.0:1789760789.550296:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74141696 chunk: [74141696, 74145792) 74145792, 4096 00020000:00200000:0.0:1789760789.550299:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.550316:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18101, 18101] 00000008:00010000:0.0:1789760789.550331:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.550335:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.550345:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74141696, 74145792) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.550420:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74141696=0x46b5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.550562:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.550565:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74145792 00000020:00010000:0.0:1789760789.550574:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.550578:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.550581:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.550584:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.550586:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.550593:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.550599:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.550618:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760789.550772:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74141696<>74145792 00000100:00100000:1.0:1789760789.550861:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565547776:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.565158:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985409 gen 3 00000008:00010000:0.0:1789760789.565353:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760789.565362:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.565372:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.565398:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760789.565440:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565547776:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.565703:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18101, 18101] 00000080:00200000:2.0:1789760789.565721:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.565727:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.565744:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74145792 i_blocks 144840 00000020:00200000:2.0:1789760789.565748:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.565760:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.565796:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74145792, count: 4096 00000080:00200000:2.0:1789760789.565805:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74145792, bytes: 4096 00000080:00200000:2.0:1789760789.565819:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.565827:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.565830:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74145792, 74149888) 0 0 00000080:00200000:2.0:1789760789.565835:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.565857:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18102, 18102] 00020000:00200000:2.0:1789760789.565867:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74145792 chunk: [74145792, 74149888) 74149888, 4096 00020000:00200000:2.0:1789760789.565872:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.565894:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18102, 18102] 00000008:00010000:2.0:1789760789.565914:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.565920:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.565933:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74145792, 74149888) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.565949:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74145792=0x46b6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.566004:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.566008:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74149888 00000008:00010000:0.0:1789760789.566131:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:0.0:1789760789.566266:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74145792<>74149888 00000100:00100000:0.0:1789760789.566324:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565547904:192.168.203.147@tcp:4:dd.0 00000020:00010000:2.0:1789760789.566570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.566574:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.566578:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.566582:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.566585:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.566594:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.566602:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:0.0:1789760789.578213:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985410 gen 3 00000008:00010000:0.0:1789760789.578267:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:0.0:1789760789.578275:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.578287:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.578311:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:0.0:1789760789.578342:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565547904:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.578676:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18102, 18102] 00000080:00200000:2.0:1789760789.578693:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.578699:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.578716:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74149888 i_blocks 144848 00000020:00200000:2.0:1789760789.578720:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.578731:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.578766:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74149888, count: 4096 00000080:00200000:2.0:1789760789.578774:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74149888, bytes: 4096 00000080:00200000:2.0:1789760789.578788:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.578796:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.578799:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74149888, 74153984) 0 0 00000080:00200000:2.0:1789760789.578803:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.578822:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18103, 18103] 00020000:00200000:2.0:1789760789.578832:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74149888 chunk: [74149888, 74153984) 74153984, 4096 00020000:00200000:2.0:1789760789.578837:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.578857:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18103, 18103] 00000008:00010000:2.0:1789760789.578877:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.578883:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.578895:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74149888, 74153984) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.578909:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74149888=0x46b7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.580132:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.580141:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74153984 00000020:00010000:2.0:1789760789.580155:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.580160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.580163:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.580167:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.580170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.580182:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.580192:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.580223:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.580365:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 74149888<>74153984 00000100:00100000:2.0:1789760789.580442:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760789.580466:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565548032:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.595498:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985411 gen 3 00000008:00010000:3.0:1789760789.595550:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760789.595560:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.595573:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.595599:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760789.595631:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565548032:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.595907:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18103, 18103] 00000080:00200000:0.0:1789760789.595926:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.596010:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.596029:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74153984 i_blocks 144856 00000020:00200000:0.0:1789760789.596033:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.596043:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.596077:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74153984, count: 4096 00000080:00200000:0.0:1789760789.596085:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74153984, bytes: 4096 00000080:00200000:0.0:1789760789.596096:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.596104:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.596106:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74153984, 74158080) 0 0 00000080:00200000:0.0:1789760789.596111:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.596128:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18104, 18104] 00020000:00200000:0.0:1789760789.596136:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74153984 chunk: [74153984, 74158080) 74158080, 4096 00020000:00200000:0.0:1789760789.596141:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.596159:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18104, 18104] 00000008:00010000:0.0:1789760789.596177:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.596183:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.596195:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74153984, 74158080) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.596208:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74153984=0x46b8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.596268:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.596271:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74158080 00000020:00010000:0.0:1789760789.596278:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.596282:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.596285:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.596289:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.596291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.596298:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.596304:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.596321:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760789.596462:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74153984<>74158080 00000100:00100000:0.0:1789760789.596580:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760789.596604:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565548160:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.606180:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985412 gen 3 00000008:00010000:0.0:1789760789.606235:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760789.606244:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.606255:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.606275:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760789.606306:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565548160:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.606570:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18104, 18104] 00000080:00200000:1.0:1789760789.606587:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.606594:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.606609:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74158080 i_blocks 144864 00000020:00200000:1.0:1789760789.606613:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.606624:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.606709:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74158080, count: 4096 00000080:00200000:1.0:1789760789.606719:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74158080, bytes: 4096 00000080:00200000:1.0:1789760789.606733:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.606741:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.606744:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74158080, 74162176) 0 0 00000080:00200000:1.0:1789760789.606760:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.606779:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18105, 18105] 00020000:00200000:1.0:1789760789.606788:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74158080 chunk: [74158080, 74162176) 74162176, 4096 00020000:00200000:1.0:1789760789.606793:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.606816:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18105, 18105] 00000008:00010000:1.0:1789760789.606835:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.606842:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.606855:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74158080, 74162176) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.606869:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74158080=0x46b9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.606931:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.606934:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74162176 00000020:00010000:1.0:1789760789.606943:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.606948:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.606951:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.606954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.606957:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.606964:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.606971:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.606988:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760789.607115:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 74158080<>74162176 00000100:00100000:1.0:1789760789.607151:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760789.607167:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565548288:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.621565:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985413 gen 3 00000008:00010000:0.0:1789760789.621618:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760789.621627:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.621637:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.622052:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760789.622087:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565548288:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.623956:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18105, 18105] 00000080:00200000:1.0:1789760789.623980:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.623986:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.624005:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74162176 i_blocks 144872 00000020:00200000:1.0:1789760789.624010:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.624026:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.624059:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74162176, count: 4096 00000080:00200000:1.0:1789760789.624067:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74162176, bytes: 4096 00000080:00200000:1.0:1789760789.624082:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.624091:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.624093:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74162176, 74166272) 0 0 00000080:00200000:1.0:1789760789.624098:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.624115:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18106, 18106] 00020000:00200000:1.0:1789760789.624125:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74162176 chunk: [74162176, 74166272) 74166272, 4096 00020000:00200000:1.0:1789760789.624130:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.624153:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18106, 18106] 00000008:00010000:1.0:1789760789.624172:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.624178:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.624190:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74162176, 74166272) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.624205:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74162176=0x46ba000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.624292:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.624295:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74166272 00000020:00010000:1.0:1789760789.624304:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.624308:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.624310:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.624314:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.624317:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.624326:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.624333:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.624562:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760789.624739:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 74162176<>74166272 00000100:00100000:0.0:1789760789.625187:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760789.625216:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565548416:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.647669:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985414 gen 3 00000008:00010000:0.0:1789760789.647715:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760789.647723:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.647734:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.647746:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760789.647769:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565548416:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.648133:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18106, 18106] 00000080:00200000:1.0:1789760789.648151:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.648158:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.648173:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74166272 i_blocks 144880 00000020:00200000:1.0:1789760789.648176:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.648186:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.648220:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74166272, count: 4096 00000080:00200000:1.0:1789760789.648227:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74166272, bytes: 4096 00000080:00200000:1.0:1789760789.648239:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.648248:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.648251:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74166272, 74170368) 0 0 00000080:00200000:1.0:1789760789.648255:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.648273:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18107, 18107] 00020000:00200000:1.0:1789760789.648282:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74166272 chunk: [74166272, 74170368) 74170368, 4096 00020000:00200000:1.0:1789760789.648286:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.648305:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18107, 18107] 00000008:00010000:1.0:1789760789.648322:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.648327:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.648339:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74166272, 74170368) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.648353:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74166272=0x46bb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.648438:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.648441:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74170368 00000020:00010000:1.0:1789760789.648448:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.648452:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.648455:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.648459:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.648461:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.648469:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.648475:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.648489:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760789.648618:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 74166272<>74170368 00000100:00100000:0.0:1789760789.648910:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760789.648931:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565548544:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.669168:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985415 gen 3 00000008:00010000:0.0:1789760789.669220:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760789.669229:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.669241:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.669316:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760789.669349:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565548544:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.672593:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18107, 18107] 00000080:00200000:1.0:1789760789.672621:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.672628:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.672689:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74170368 i_blocks 144888 00000020:00200000:1.0:1789760789.672696:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.672710:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.672743:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74170368, count: 4096 00000080:00200000:1.0:1789760789.672753:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74170368, bytes: 4096 00000080:00200000:1.0:1789760789.672767:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.672855:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.672858:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74170368, 74174464) 0 0 00000080:00200000:1.0:1789760789.672862:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.672880:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18108, 18108] 00020000:00200000:1.0:1789760789.672890:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74170368 chunk: [74170368, 74174464) 74174464, 4096 00020000:00200000:1.0:1789760789.672894:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.672916:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18108, 18108] 00000008:00010000:1.0:1789760789.672998:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.673005:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.673067:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74170368, 74174464) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.673082:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74170368=0x46bc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.673145:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.673149:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74174464 00000020:00010000:1.0:1789760789.673157:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.673160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.673163:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.673167:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.673170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.673179:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.673188:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.673207:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760789.673334:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 74170368<>74174464 00000100:00100000:0.0:1789760789.673489:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565548672:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.688275:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985416 gen 3 00000008:00010000:0.0:1789760789.688325:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760789.688334:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.688345:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.688474:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760789.688506:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565548672:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.689279:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18108, 18108] 00000080:00200000:3.0:1789760789.689299:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.689305:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760789.689322:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74174464 i_blocks 144896 00000020:00200000:3.0:1789760789.689326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760789.689339:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760789.689446:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74174464, count: 4096 00000080:00200000:3.0:1789760789.689454:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74174464, bytes: 4096 00000080:00200000:3.0:1789760789.689468:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760789.689477:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760789.689479:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74174464, 74178560) 0 0 00000080:00200000:3.0:1789760789.689571:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760789.689594:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18109, 18109] 00020000:00200000:3.0:1789760789.689603:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74174464 chunk: [74174464, 74178560) 74178560, 4096 00020000:00200000:3.0:1789760789.689608:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760789.689630:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18109, 18109] 00000008:00010000:3.0:1789760789.689669:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760789.689676:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760789.689690:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74174464, 74178560) append 0 append lockpos 0 00000080:00200000:3.0:1789760789.689706:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74174464=0x46bd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760789.689849:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760789.689853:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74178560 00000020:00010000:3.0:1789760789.689863:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760789.689867:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.689870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.689874:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760789.689877:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760789.689887:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760789.689894:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760789.689919:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760789.690137:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 74174464<>74178560 00000100:00100000:3.0:1789760789.690197:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760789.690219:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565548800:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.706218:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985417 gen 3 00000008:00010000:3.0:1789760789.706274:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:3.0:1789760789.706285:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.706298:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.706315:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:3.0:1789760789.706349:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565548800:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.706723:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18109, 18109] 00000080:00200000:0.0:1789760789.706741:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.706747:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.706763:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74178560 i_blocks 144904 00000020:00200000:0.0:1789760789.706767:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.706788:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.706822:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74178560, count: 4096 00000080:00200000:0.0:1789760789.706830:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74178560, bytes: 4096 00000080:00200000:0.0:1789760789.706845:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.706854:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.706857:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74178560, 74182656) 0 0 00000080:00200000:0.0:1789760789.706861:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.706878:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18110, 18110] 00020000:00200000:0.0:1789760789.706886:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74178560 chunk: [74178560, 74182656) 74182656, 4096 00020000:00200000:0.0:1789760789.706891:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.706911:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18110, 18110] 00000008:00010000:0.0:1789760789.706929:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.706936:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.707192:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74178560, 74182656) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.707207:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74178560=0x46be000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.707269:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.707272:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74182656 00000020:00010000:0.0:1789760789.707281:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.707285:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.707288:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.707291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.707294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.707301:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.707308:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.707327:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760789.707591:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74178560<>74182656 00000100:00100000:0.0:1789760789.707922:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760789.708130:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565548928:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.718608:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985418 gen 3 00000008:00010000:0.0:1789760789.718700:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760789.718710:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.718720:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.718732:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000080:00200000:1.0:1789760789.718912:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18110, 18110] 00000080:00200000:1.0:1789760789.718930:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.718937:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.718951:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74182656 i_blocks 144912 00000020:00200000:1.0:1789760789.718955:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.718966:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.718998:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74182656, count: 4096 00000080:00200000:1.0:1789760789.719008:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74182656, bytes: 4096 00000080:00200000:1.0:1789760789.719019:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.719028:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.719030:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74182656, 74186752) 0 0 00000080:00200000:1.0:1789760789.719035:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.719052:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18111, 18111] 00020000:00200000:1.0:1789760789.719061:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74182656 chunk: [74182656, 74186752) 74186752, 4096 00020000:00200000:1.0:1789760789.719067:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.719087:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18111, 18111] 00000008:00010000:1.0:1789760789.719105:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.719111:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.719123:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74182656, 74186752) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.719137:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74182656=0x46bf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.719184:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.719187:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74186752 00000020:00010000:1.0:1789760789.719194:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.719198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.719201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.719204:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.719207:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.719213:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.719221:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:0.0:1789760789.719278:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565548928:192.168.203.147@tcp:4:dd.0 00000008:00010000:0.0:1789760789.719322:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:0.0:1789760789.719575:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74182656<>74186752 00000100:00100000:0.0:1789760789.719628:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565549056:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.732601:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985419 gen 3 00000008:00010000:0.0:1789760789.732686:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760789.732695:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.732706:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.732936:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760789.732967:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565549056:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.739733:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18111, 18111] 00000080:00200000:1.0:1789760789.739762:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.739769:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.739789:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74186752 i_blocks 144920 00000020:00200000:1.0:1789760789.739795:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.739812:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.739849:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74186752, count: 4096 00000080:00200000:1.0:1789760789.739858:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74186752, bytes: 4096 00000080:00200000:1.0:1789760789.739873:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.739882:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.739884:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74186752, 74190848) 0 0 00000080:00200000:1.0:1789760789.739889:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.739907:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18112, 18112] 00020000:00200000:1.0:1789760789.739921:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74186752 chunk: [74186752, 74190848) 74190848, 4096 00020000:00200000:1.0:1789760789.739926:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.739949:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18112, 18112] 00000008:00010000:1.0:1789760789.739967:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.739973:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.739984:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74186752, 74190848) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.740002:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74186752=0x46c0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.740077:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.740081:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74190848 00000020:00010000:1.0:1789760789.740088:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.740091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.740094:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.740098:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.740100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.740108:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.740114:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.740143:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760789.740263:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 74186752<>74190848 00000100:00100000:0.0:1789760789.740419:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565549184:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.750587:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985420 gen 3 00000008:00010000:0.0:1789760789.750640:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760789.750740:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.750751:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.750810:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760789.750842:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565549184:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.750938:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18112, 18112] 00000080:00200000:1.0:1789760789.750954:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.750961:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.751061:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74190848 i_blocks 144928 00000020:00200000:0.0:1789760789.751066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.751076:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.751109:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74190848, count: 4096 00000080:00200000:0.0:1789760789.751116:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74190848, bytes: 4096 00000080:00200000:0.0:1789760789.751127:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.751137:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.751141:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74190848, 74194944) 0 0 00000080:00200000:0.0:1789760789.751146:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.751164:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18113, 18113] 00020000:00200000:0.0:1789760789.751173:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74190848 chunk: [74190848, 74194944) 74194944, 4096 00020000:00200000:0.0:1789760789.751178:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.751198:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18113, 18113] 00000008:00010000:0.0:1789760789.751217:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.751224:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.751236:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74190848, 74194944) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.751251:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74190848=0x46c1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.751306:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.751309:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74194944 00000020:00010000:0.0:1789760789.751317:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.751321:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.751324:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.751328:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.751331:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.751339:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.751346:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.751361:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760789.751698:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74190848<>74194944 00000100:00100000:1.0:1789760789.751772:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565549312:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.763592:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985421 gen 3 00000008:00010000:0.0:1789760789.764421:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760789.764442:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.764453:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.764544:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760789.764577:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565549312:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.764795:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18113, 18113] 00000080:00200000:3.0:1789760789.764813:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.765182:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760789.765201:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74194944 i_blocks 144936 00000020:00200000:3.0:1789760789.765205:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760789.765219:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760789.765254:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74194944, count: 4096 00000080:00200000:3.0:1789760789.765262:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74194944, bytes: 4096 00000080:00200000:3.0:1789760789.765275:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760789.765283:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760789.765286:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74194944, 74199040) 0 0 00000080:00200000:3.0:1789760789.765290:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760789.765307:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18114, 18114] 00020000:00200000:3.0:1789760789.765315:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74194944 chunk: [74194944, 74199040) 74199040, 4096 00020000:00200000:3.0:1789760789.765319:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760789.765339:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18114, 18114] 00000008:00010000:3.0:1789760789.765356:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760789.765362:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760789.765374:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74194944, 74199040) append 0 append lockpos 0 00000080:00200000:3.0:1789760789.765388:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74194944=0x46c2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760789.766108:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760789.766114:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74199040 00000020:00010000:3.0:1789760789.766124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760789.766128:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.766131:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.766135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760789.766138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760789.766148:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760789.766155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760789.766184:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760789.766317:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 74194944<>74199040 00000100:00100000:2.0:1789760789.766483:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565549440:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.782816:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985422 gen 3 00000008:00010000:2.0:1789760789.782870:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760789.782880:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.782892:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.782915:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760789.782947:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565549440:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.783115:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18114, 18114] 00000080:00200000:3.0:1789760789.783133:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.783139:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760789.783165:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74199040 i_blocks 144944 00000020:00200000:3.0:1789760789.783169:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760789.783178:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760789.783212:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74199040, count: 4096 00000080:00200000:3.0:1789760789.783219:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74199040, bytes: 4096 00000080:00200000:3.0:1789760789.783229:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760789.783238:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760789.783240:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74199040, 74203136) 0 0 00000080:00200000:3.0:1789760789.783245:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760789.783260:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18115, 18115] 00020000:00200000:3.0:1789760789.783268:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74199040 chunk: [74199040, 74203136) 74203136, 4096 00020000:00200000:3.0:1789760789.783273:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760789.783291:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18115, 18115] 00000008:00010000:3.0:1789760789.783307:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760789.783313:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760789.783326:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74199040, 74203136) append 0 append lockpos 0 00000080:00200000:3.0:1789760789.783339:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74199040=0x46c3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760789.783497:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760789.783501:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74203136 00000020:00010000:3.0:1789760789.783509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760789.783513:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.783517:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.783520:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760789.783524:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760789.783532:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760789.783539:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760789.783556:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760789.783706:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 74199040<>74203136 00000100:00100000:2.0:1789760789.784101:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565549568:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.797713:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985423 gen 3 00000008:00010000:2.0:1789760789.797764:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760789.797783:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.797795:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.797815:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760789.797845:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565549568:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.798013:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18115, 18115] 00000080:00200000:3.0:1789760789.798030:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.798035:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760789.798051:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74203136 i_blocks 144952 00000020:00200000:3.0:1789760789.798055:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760789.798065:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760789.798098:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74203136, count: 4096 00000080:00200000:3.0:1789760789.798106:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74203136, bytes: 4096 00000080:00200000:3.0:1789760789.798117:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760789.798125:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760789.798127:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74203136, 74207232) 0 0 00000080:00200000:3.0:1789760789.798132:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760789.798157:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18116, 18116] 00020000:00200000:3.0:1789760789.798166:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74203136 chunk: [74203136, 74207232) 74207232, 4096 00020000:00200000:3.0:1789760789.798170:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760789.798188:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18116, 18116] 00000008:00010000:3.0:1789760789.798205:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760789.798211:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760789.798221:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74203136, 74207232) append 0 append lockpos 0 00000080:00200000:3.0:1789760789.798234:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74203136=0x46c4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760789.798287:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760789.798289:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74207232 00000020:00010000:3.0:1789760789.798296:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760789.798300:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.798303:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.798307:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760789.798311:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760789.798318:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760789.798324:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760789.798339:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760789.798466:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 74203136<>74207232 00000100:00100000:2.0:1789760789.798718:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760789.798734:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565549696:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.807903:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985424 gen 3 00000008:00010000:2.0:1789760789.807956:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760789.807964:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.807978:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.808009:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760789.808039:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565549696:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.808260:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18116, 18116] 00000080:00200000:3.0:1789760789.808277:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.808283:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760789.808299:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74207232 i_blocks 144960 00000020:00200000:3.0:1789760789.808303:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760789.808313:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760789.808350:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74207232, count: 4096 00000080:00200000:3.0:1789760789.808357:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74207232, bytes: 4096 00000080:00200000:3.0:1789760789.808367:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760789.808376:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760789.808378:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74207232, 74211328) 0 0 00000080:00200000:3.0:1789760789.808382:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760789.808399:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18117, 18117] 00020000:00200000:3.0:1789760789.808410:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74207232 chunk: [74207232, 74211328) 74211328, 4096 00020000:00200000:3.0:1789760789.808414:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760789.808446:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18117, 18117] 00000008:00010000:3.0:1789760789.808465:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760789.808471:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760789.808484:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74207232, 74211328) append 0 append lockpos 0 00000080:00200000:3.0:1789760789.808498:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74207232=0x46c5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760789.808561:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760789.808564:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74211328 00000020:00010000:3.0:1789760789.808572:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760789.808576:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.808580:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.808583:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760789.808587:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760789.808596:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760789.808604:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760789.808622:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760789.808778:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 74207232<>74211328 00000100:00100000:2.0:1789760789.809101:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760789.809124:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565549824:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.820238:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985425 gen 3 00000008:00010000:2.0:1789760789.820290:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760789.820298:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.820310:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.820328:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760789.820358:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565549824:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.820713:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18117, 18117] 00000080:00200000:3.0:1789760789.820730:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.820735:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760789.820750:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74211328 i_blocks 144968 00000020:00200000:3.0:1789760789.820754:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760789.820765:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760789.820801:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74211328, count: 4096 00000080:00200000:3.0:1789760789.820809:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74211328, bytes: 4096 00000080:00200000:3.0:1789760789.820820:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760789.820829:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760789.820832:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74211328, 74215424) 0 0 00000080:00200000:3.0:1789760789.820836:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760789.820855:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18118, 18118] 00020000:00200000:3.0:1789760789.820864:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74211328 chunk: [74211328, 74215424) 74215424, 4096 00020000:00200000:3.0:1789760789.820868:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760789.820889:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18118, 18118] 00000008:00010000:3.0:1789760789.820907:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760789.820913:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760789.820925:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74211328, 74215424) append 0 append lockpos 0 00000080:00200000:3.0:1789760789.820941:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74211328=0x46c6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760789.821074:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760789.821077:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74215424 00000020:00010000:3.0:1789760789.821087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760789.821091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.821094:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.821099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760789.821103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760789.821111:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760789.821119:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760789.821138:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760789.821971:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 74211328<>74215424 00000100:00100000:2.0:1789760789.822512:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760789.822539:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565549952:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.833212:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985426 gen 3 00000008:00010000:2.0:1789760789.833261:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760789.833270:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.833282:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.833302:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760789.833332:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565549952:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.833501:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18118, 18118] 00000080:00200000:3.0:1789760789.833517:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.833522:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760789.833537:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74215424 i_blocks 144976 00000020:00200000:3.0:1789760789.833540:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760789.833551:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760789.833588:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74215424, count: 4096 00000080:00200000:3.0:1789760789.833596:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74215424, bytes: 4096 00000080:00200000:3.0:1789760789.833606:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760789.833614:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760789.833616:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74215424, 74219520) 0 0 00000080:00200000:3.0:1789760789.833620:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760789.833638:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18119, 18119] 00020000:00200000:3.0:1789760789.833688:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74215424 chunk: [74215424, 74219520) 74219520, 4096 00020000:00200000:3.0:1789760789.833693:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760789.833712:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18119, 18119] 00000008:00010000:3.0:1789760789.833729:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760789.833735:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760789.833746:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74215424, 74219520) append 0 append lockpos 0 00000080:00200000:3.0:1789760789.833758:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74215424=0x46c7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760789.833805:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760789.833808:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74219520 00000020:00010000:3.0:1789760789.833816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760789.833819:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.833822:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.833826:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760789.833829:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760789.833836:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760789.833842:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.833919:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:2.0:1789760789.834303:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 74215424<>74219520 00000100:00100000:2.0:1789760789.834629:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565550080:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.847210:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985427 gen 3 00000008:00010000:2.0:1789760789.847260:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760789.847269:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.847281:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.847304:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760789.847336:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565550080:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.848025:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18119, 18119] 00000080:00200000:3.0:1789760789.848046:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.848052:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760789.848070:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74219520 i_blocks 144984 00000020:00200000:3.0:1789760789.848075:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760789.848087:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760789.848127:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74219520, count: 4096 00000080:00200000:3.0:1789760789.848136:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74219520, bytes: 4096 00000080:00200000:3.0:1789760789.848162:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760789.848171:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760789.848174:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74219520, 74223616) 0 0 00000080:00200000:3.0:1789760789.848179:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760789.848198:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18120, 18120] 00020000:00200000:3.0:1789760789.848208:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74219520 chunk: [74219520, 74223616) 74223616, 4096 00020000:00200000:3.0:1789760789.848213:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760789.848235:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18120, 18120] 00000008:00010000:3.0:1789760789.848259:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760789.848267:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760789.848284:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74219520, 74223616) append 0 append lockpos 0 00000080:00200000:3.0:1789760789.848301:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74219520=0x46c8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760789.848384:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760789.848387:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74223616 00000020:00010000:3.0:1789760789.848395:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760789.848398:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.848402:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.848406:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760789.848409:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760789.848418:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760789.848435:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760789.848458:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760789.848615:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 74219520<>74223616 00000100:00100000:2.0:1789760789.849864:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565550208:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.863617:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985428 gen 3 00000008:00010000:2.0:1789760789.863708:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760789.863717:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.863731:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.863752:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760789.863790:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565550208:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.864315:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18120, 18120] 00000080:00200000:3.0:1789760789.864333:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.864339:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760789.864355:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74223616 i_blocks 144992 00000020:00200000:3.0:1789760789.864371:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760789.864382:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760789.864417:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74223616, count: 4096 00000080:00200000:3.0:1789760789.864434:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74223616, bytes: 4096 00000080:00200000:3.0:1789760789.864446:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760789.864455:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760789.864457:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74223616, 74227712) 0 0 00000080:00200000:3.0:1789760789.864462:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760789.864479:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18121, 18121] 00020000:00200000:3.0:1789760789.864498:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74223616 chunk: [74223616, 74227712) 74227712, 4096 00020000:00200000:3.0:1789760789.864503:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760789.864525:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18121, 18121] 00000008:00010000:3.0:1789760789.864544:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760789.864555:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760789.864567:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74223616, 74227712) append 0 append lockpos 0 00000080:00200000:3.0:1789760789.864581:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74223616=0x46c9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760789.864679:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760789.864683:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74227712 00000020:00010000:3.0:1789760789.864692:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760789.864696:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.864699:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.864702:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760789.864705:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760789.864714:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760789.864721:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760789.864740:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760789.864869:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 74223616<>74227712 00000100:00100000:2.0:1789760789.865513:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760789.865537:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565550336:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.875301:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985429 gen 3 00000008:00010000:2.0:1789760789.875363:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760789.875385:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.875398:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.875430:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760789.875463:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565550336:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.875582:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18121, 18121] 00000080:00200000:3.0:1789760789.875601:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.875609:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760789.875627:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74227712 i_blocks 145000 00000020:00200000:3.0:1789760789.875631:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760789.875689:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760789.875731:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74227712, count: 4096 00000080:00200000:3.0:1789760789.875739:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74227712, bytes: 4096 00000080:00200000:3.0:1789760789.875751:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760789.875761:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760789.875763:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74227712, 74231808) 0 0 00000080:00200000:3.0:1789760789.875780:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760789.875798:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18122, 18122] 00020000:00200000:3.0:1789760789.875807:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74227712 chunk: [74227712, 74231808) 74231808, 4096 00020000:00200000:3.0:1789760789.875812:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760789.875830:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18122, 18122] 00000008:00010000:3.0:1789760789.875848:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760789.875854:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760789.875867:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74227712, 74231808) append 0 append lockpos 0 00000080:00200000:3.0:1789760789.875882:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74227712=0x46ca000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760789.875944:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760789.875947:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74231808 00000020:00010000:3.0:1789760789.875955:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:3.0:1789760789.875958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.875962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760789.875965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760789.875968:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760789.875976:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760789.876112:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760789.876131:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760789.876331:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 74227712<>74231808 00000100:00100000:2.0:1789760789.876907:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760789.876936:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565550464:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760789.887391:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985430 gen 3 00000008:00010000:2.0:1789760789.887451:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760789.887460:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760789.887472:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760789.887581:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760789.887612:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565550464:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760789.887951:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18122, 18122] 00000080:00200000:3.0:1789760789.887968:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760789.887975:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.888226:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74231808 i_blocks 145008 00000020:00200000:2.0:1789760789.888231:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.888243:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.888281:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74231808, count: 4096 00000080:00200000:2.0:1789760789.888290:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74231808, bytes: 4096 00000080:00200000:2.0:1789760789.888303:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.888313:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.888316:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74231808, 74235904) 0 0 00000080:00200000:2.0:1789760789.888321:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.888339:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18123, 18123] 00020000:00200000:2.0:1789760789.888346:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74231808 chunk: [74231808, 74235904) 74235904, 4096 00020000:00200000:2.0:1789760789.888351:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.888435:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18123, 18123] 00000008:00010000:2.0:1789760789.888455:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.888461:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.888473:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74231808, 74235904) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.888547:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74231808=0x46cb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.888613:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.888616:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74235904 00000020:00010000:2.0:1789760789.888626:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.888629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.888632:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.888635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.888638:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.888670:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.888677:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.888735:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f844189840 00000008:00100000:2.0:1789760789.888923:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 74231808<>74235904 00000100:00100000:3.0:1789760789.889067:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565550592:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.896898:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985431 gen 3 00000008:00010000:3.0:1789760789.896950:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f844189840 00000020:00200000:3.0:1789760789.896960:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.896970:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.897001:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f844189840 00000100:00100000:3.0:1789760789.897033:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565550592:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.897124:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18123, 18123] 00000080:00200000:2.0:1789760789.897151:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.897157:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.897173:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74235904 i_blocks 145016 00000020:00200000:2.0:1789760789.897177:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.897187:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.897223:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74235904, count: 4096 00000080:00200000:2.0:1789760789.897231:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74235904, bytes: 4096 00000080:00200000:2.0:1789760789.897242:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.897251:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.897253:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74235904, 74240000) 0 0 00000080:00200000:2.0:1789760789.897258:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.897276:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18124, 18124] 00020000:00200000:2.0:1789760789.897286:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74235904 chunk: [74235904, 74240000) 74240000, 4096 00020000:00200000:2.0:1789760789.897291:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.897314:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18124, 18124] 00000008:00010000:2.0:1789760789.897331:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.897338:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.897350:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74235904, 74240000) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.897364:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74235904=0x46cc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.897436:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.897439:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74240000 00000020:00010000:2.0:1789760789.897448:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.897451:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.897463:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.897467:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.897470:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.897478:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.897485:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.897503:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760789.897628:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 74235904<>74240000 00000100:00100000:3.0:1789760789.897708:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565550720:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.906855:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985432 gen 3 00000008:00010000:3.0:1789760789.906907:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:3.0:1789760789.906916:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.906928:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.906984:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:3.0:1789760789.907016:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565550720:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.907020:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18124, 18124] 00000080:00200000:2.0:1789760789.907037:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.907043:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.907059:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74240000 i_blocks 145024 00000020:00200000:2.0:1789760789.907063:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.907073:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.907109:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74240000, count: 4096 00000080:00200000:2.0:1789760789.907117:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74240000, bytes: 4096 00000080:00200000:2.0:1789760789.907127:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.907136:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.907139:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74240000, 74244096) 0 0 00000080:00200000:2.0:1789760789.907143:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.907162:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18125, 18125] 00020000:00200000:2.0:1789760789.907172:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74240000 chunk: [74240000, 74244096) 74244096, 4096 00020000:00200000:2.0:1789760789.907177:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.907257:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18125, 18125] 00000008:00010000:2.0:1789760789.907281:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.907287:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.907300:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74240000, 74244096) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.907313:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74240000=0x46cd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.907436:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.907440:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74244096 00000020:00010000:2.0:1789760789.907449:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.907453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.907456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.907460:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.907464:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.907472:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.907478:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.907494:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760789.907613:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 74240000<>74244096 00000100:00100000:3.0:1789760789.907887:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760789.907904:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565550848:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.919112:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985433 gen 3 00000008:00010000:3.0:1789760789.919340:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760789.919350:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.919367:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.919390:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760789.919430:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565550848:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760789.919457:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18125, 18125] 00000080:00200000:2.0:1789760789.919475:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760789.919481:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760789.919505:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74244096 i_blocks 145032 00000020:00200000:2.0:1789760789.919509:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760789.919519:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760789.919553:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74244096, count: 4096 00000080:00200000:2.0:1789760789.919560:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74244096, bytes: 4096 00000080:00200000:2.0:1789760789.919571:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760789.919580:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760789.919582:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74244096, 74248192) 0 0 00000080:00200000:2.0:1789760789.919587:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760789.919604:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18126, 18126] 00020000:00200000:2.0:1789760789.919613:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74244096 chunk: [74244096, 74248192) 74248192, 4096 00020000:00200000:2.0:1789760789.919618:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760789.919637:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18126, 18126] 00000008:00010000:2.0:1789760789.919690:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760789.919697:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760789.919710:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74244096, 74248192) append 0 append lockpos 0 00000080:00200000:2.0:1789760789.919723:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74244096=0x46ce000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760789.919792:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760789.919795:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74248192 00000020:00010000:2.0:1789760789.919803:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:2.0:1789760789.919807:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.919810:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760789.919814:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760789.919825:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760789.919834:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760789.919840:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760789.919856:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760789.919986:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 74244096<>74248192 00000100:00100000:3.0:1789760789.920041:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565550976:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760789.940541:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985434 gen 3 00000008:00010000:3.0:1789760789.940598:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:3.0:1789760789.940606:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760789.940619:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760789.940639:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:3.0:1789760789.940705:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565550976:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760789.940913:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18126, 18126] 00000080:00200000:0.0:1789760789.940932:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760789.940938:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760789.940955:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74248192 i_blocks 145040 00000020:00200000:0.0:1789760789.940959:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760789.940970:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760789.941010:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74248192, count: 4096 00000080:00200000:0.0:1789760789.941018:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74248192, bytes: 4096 00000080:00200000:0.0:1789760789.941034:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760789.941043:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760789.941046:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74248192, 74252288) 0 0 00000080:00200000:0.0:1789760789.941051:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760789.941068:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18127, 18127] 00020000:00200000:0.0:1789760789.941078:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74248192 chunk: [74248192, 74252288) 74252288, 4096 00020000:00200000:0.0:1789760789.941082:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760789.941103:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18127, 18127] 00000008:00010000:0.0:1789760789.941121:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760789.941128:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760789.941140:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74248192, 74252288) append 0 append lockpos 0 00000080:00200000:0.0:1789760789.941155:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74248192=0x46cf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760789.941441:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760789.941445:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74252288 00000020:00010000:0.0:1789760789.941453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:0.0:1789760789.941457:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.941460:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760789.941464:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760789.941467:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760789.941475:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760789.941482:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760789.941501:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760789.941635:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74248192<>74252288 00000100:00100000:1.0:1789760789.942578:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760789.942965:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565551104:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.954127:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985435 gen 3 00000008:00010000:0.0:1789760789.954179:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760789.954188:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.954200:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.954221:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760789.954251:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565551104:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.954432:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18127, 18127] 00000080:00200000:1.0:1789760789.954449:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.956129:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.956266:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74252288 i_blocks 145048 00000020:00200000:1.0:1789760789.956274:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.956293:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.956400:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74252288, count: 4096 00000080:00200000:1.0:1789760789.956411:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74252288, bytes: 4096 00000080:00200000:1.0:1789760789.956435:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.956444:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.956447:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74252288, 74256384) 0 0 00000080:00200000:1.0:1789760789.956459:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.956477:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18128, 18128] 00020000:00200000:1.0:1789760789.956578:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74252288 chunk: [74252288, 74256384) 74256384, 4096 00020000:00200000:1.0:1789760789.956584:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.956608:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18128, 18128] 00000008:00010000:1.0:1789760789.956627:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.956633:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.956667:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74252288, 74256384) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.956682:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74252288=0x46d0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.956871:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.956874:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74256384 00000020:00010000:1.0:1789760789.956883:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.956888:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.956890:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.956894:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.956897:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.956906:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.956913:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.957046:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760789.957292:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 74252288<>74256384 00000100:00100000:0.0:1789760789.957632:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760789.957876:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565551232:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.975785:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985436 gen 3 00000008:00010000:0.0:1789760789.975835:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760789.975844:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.975857:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.975879:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760789.975911:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565551232:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760789.976170:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18128, 18128] 00000080:00200000:1.0:1789760789.976187:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760789.976193:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760789.976210:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74256384 i_blocks 145056 00000020:00200000:1.0:1789760789.976214:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760789.976223:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760789.976258:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74256384, count: 4096 00000080:00200000:1.0:1789760789.976265:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74256384, bytes: 4096 00000080:00200000:1.0:1789760789.976275:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760789.976284:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760789.976286:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74256384, 74260480) 0 0 00000080:00200000:1.0:1789760789.976290:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760789.976306:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18129, 18129] 00020000:00200000:1.0:1789760789.976315:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74256384 chunk: [74256384, 74260480) 74260480, 4096 00020000:00200000:1.0:1789760789.976331:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760789.976351:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18129, 18129] 00000008:00010000:1.0:1789760789.976367:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760789.976372:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760789.976384:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74256384, 74260480) append 0 append lockpos 0 00000080:00200000:1.0:1789760789.976397:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74256384=0x46d1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760789.976460:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760789.976463:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74260480 00000020:00010000:1.0:1789760789.976470:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_release():245 00000020:00010000:1.0:1789760789.976474:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.976477:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760789.976480:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0b78 (ffff93f84418a098/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760789.976483:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760789.976599:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760789.976607:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760789.976629:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760789.976855:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 74256384<>74260480 00000100:00100000:1.0:1789760789.976892:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760789.976909:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565551360:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760789.998798:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985437 gen 3 00000008:00010000:0.0:1789760789.998849:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760789.998858:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760789.998870:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760789.998896:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760789.998926:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565551360:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760790.000504:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18129, 18129] 00000080:00200000:3.0:1789760790.000528:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760790.000535:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760790.000554:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74260480 i_blocks 145064 00000020:00200000:3.0:1789760790.000558:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760790.000571:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760790.000640:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74260480, count: 4096 00000080:00200000:3.0:1789760790.000682:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74260480, bytes: 4096 00000080:00200000:3.0:1789760790.000696:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760790.000704:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760790.000707:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74260480, 74264576) 0 0 00000080:00200000:3.0:1789760790.000711:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760790.000727:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18130, 18130] 00020000:00200000:3.0:1789760790.000736:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74260480 chunk: [74260480, 74264576) 74264576, 4096 00020000:00200000:3.0:1789760790.000741:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760790.000763:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18130, 18130] 00000008:00010000:3.0:1789760790.000782:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760790.000789:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760790.000801:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74260480, 74264576) append 0 append lockpos 0 00000080:00200000:3.0:1789760790.000818:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74260480=0x46d2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760790.000884:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760790.000888:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74264576 00000020:00010000:3.0:1789760790.000897:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760790.000900:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760790.000903:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760790.000906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760790.000909:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760790.000918:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760790.000925:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760790.000945:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760790.001078:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 74260480<>74264576 00000100:00100000:3.0:1789760790.001139:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760790.001160:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565551488:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.013236:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985438 gen 3 00000008:00010000:3.0:1789760790.013289:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:3.0:1789760790.013297:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.013310:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.013337:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:3.0:1789760790.013377:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565551488:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.013750:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18130, 18130] 00000080:00200000:0.0:1789760790.013768:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.013774:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.013792:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74264576 i_blocks 145072 00000020:00200000:0.0:1789760790.013796:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.013808:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.013842:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74264576, count: 4096 00000080:00200000:0.0:1789760790.013850:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74264576, bytes: 4096 00000080:00200000:0.0:1789760790.013861:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.013870:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.013873:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74264576, 74268672) 0 0 00000080:00200000:0.0:1789760790.013878:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.013896:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18131, 18131] 00020000:00200000:0.0:1789760790.013905:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74264576 chunk: [74264576, 74268672) 74268672, 4096 00020000:00200000:0.0:1789760790.013910:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.013931:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18131, 18131] 00000008:00010000:0.0:1789760790.013951:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.013957:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.013969:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74264576, 74268672) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.013985:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74264576=0x46d3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.014045:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.014048:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74268672 00000020:00010000:0.0:1789760790.014057:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.014060:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.014064:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.014067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.014070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.014078:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.014085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.014102:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760790.014220:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74264576<>74268672 00000100:00100000:0.0:1789760790.014268:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565551616:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.027532:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985439 gen 3 00000008:00010000:1.0:1789760790.027583:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760790.027592:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.027603:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.027627:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000080:00200000:0.0:1789760790.027691:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18131, 18131] 00000100:00100000:1.0:1789760790.027702:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565551616:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.027711:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.027718:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.027735:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74268672 i_blocks 145080 00000020:00200000:0.0:1789760790.027739:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.027752:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.029082:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74268672, count: 4096 00000080:00200000:2.0:1789760790.029100:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74268672, bytes: 4096 00000080:00200000:2.0:1789760790.029115:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.029124:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.029127:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74268672, 74272768) 0 0 00000080:00200000:2.0:1789760790.029133:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.029150:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18132, 18132] 00020000:00200000:2.0:1789760790.029161:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74268672 chunk: [74268672, 74272768) 74272768, 4096 00020000:00200000:2.0:1789760790.029165:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.029223:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18132, 18132] 00000008:00010000:2.0:1789760790.029241:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.029247:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.029259:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74268672, 74272768) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.029274:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74268672=0x46d4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.029531:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.029534:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74272768 00000020:00010000:2.0:1789760790.029544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.029548:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.029551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.029554:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.029557:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.029566:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.029575:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.029677:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760790.029891:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74268672<>74272768 00000100:00100000:3.0:1789760790.029997:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565551744:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.039923:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985440 gen 3 00000008:00010000:2.0:1789760790.039986:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:2.0:1789760790.039996:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.040008:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.040032:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:2.0:1789760790.040066:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565551744:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760790.040273:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18132, 18132] 00000080:00200000:3.0:1789760790.040289:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760790.040296:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760790.040312:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74272768 i_blocks 145088 00000020:00200000:3.0:1789760790.040316:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760790.040326:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760790.040623:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74272768, count: 4096 00000080:00200000:3.0:1789760790.040634:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74272768, bytes: 4096 00000080:00200000:3.0:1789760790.040742:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760790.040751:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760790.040753:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74272768, 74276864) 0 0 00000080:00200000:3.0:1789760790.040758:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.041140:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18133, 18133] 00020000:00200000:2.0:1789760790.041150:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74272768 chunk: [74272768, 74276864) 74276864, 4096 00020000:00200000:2.0:1789760790.041155:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.041178:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18133, 18133] 00000008:00010000:2.0:1789760790.041198:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.041204:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.041218:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74272768, 74276864) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.041232:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74272768=0x46d5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.041286:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.041289:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74276864 00000020:00010000:2.0:1789760790.041297:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.041301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.041304:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.041307:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.041311:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.041331:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.041338:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.041393:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760790.041625:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74272768<>74276864 00000100:00100000:2.0:1789760790.041700:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760790.041720:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565551872:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760790.044760:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760790.052177:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985441 gen 3 00000008:00010000:2.0:1789760790.052229:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:2.0:1789760790.052238:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.052249:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.052277:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:2.0:1789760790.052309:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565551872:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.052415:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18133, 18133] 00000080:00200000:0.0:1789760790.052441:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.052447:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.052464:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74276864 i_blocks 145096 00000020:00200000:0.0:1789760790.052468:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.052478:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.052520:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74276864, count: 4096 00000080:00200000:0.0:1789760790.052528:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74276864, bytes: 4096 00000080:00200000:0.0:1789760790.052538:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.052547:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.052550:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74276864, 74280960) 0 0 00000080:00200000:0.0:1789760790.052554:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.052572:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18134, 18134] 00020000:00200000:0.0:1789760790.052581:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74276864 chunk: [74276864, 74280960) 74280960, 4096 00020000:00200000:0.0:1789760790.052585:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.052608:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18134, 18134] 00000008:00010000:0.0:1789760790.052638:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.052679:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.052692:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74276864, 74280960) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.052707:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74276864=0x46d6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.052776:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.052779:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74280960 00000020:00010000:0.0:1789760790.052787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.052791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.052794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.052797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.052800:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.052808:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.052814:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.052832:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760790.052957:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 74276864<>74280960 00000100:00100000:1.0:1789760790.053033:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760790.053052:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565552000:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.065384:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985442 gen 3 00000008:00010000:1.0:1789760790.065444:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760790.065453:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.065469:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.065531:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760790.065561:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565552000:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.065800:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18134, 18134] 00000080:00200000:0.0:1789760790.065818:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.065825:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.065842:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74280960 i_blocks 145104 00000020:00200000:0.0:1789760790.065846:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.065859:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.065894:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74280960, count: 4096 00000080:00200000:0.0:1789760790.065902:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74280960, bytes: 4096 00000080:00200000:0.0:1789760790.065913:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.065922:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.065924:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74280960, 74285056) 0 0 00000080:00200000:0.0:1789760790.065929:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.065947:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18135, 18135] 00020000:00200000:0.0:1789760790.065957:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74280960 chunk: [74280960, 74285056) 74285056, 4096 00020000:00200000:0.0:1789760790.065961:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.065980:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18135, 18135] 00000008:00010000:0.0:1789760790.065998:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.066003:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.066015:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74280960, 74285056) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.066028:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74280960=0x46d7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.066087:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.066089:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74285056 00000020:00010000:0.0:1789760790.066097:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.066101:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.066103:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.066107:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.066110:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.066118:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.066124:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.066142:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760790.066275:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 74280960<>74285056 00000100:00100000:0.0:1789760790.066319:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760790.066338:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565552128:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.089392:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985443 gen 3 00000008:00010000:1.0:1789760790.089460:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760790.089469:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.089481:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.089501:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760790.089528:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565552128:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.089679:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18135, 18135] 00000080:00200000:0.0:1789760790.089696:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.089702:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.089771:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74285056 i_blocks 145112 00000020:00200000:0.0:1789760790.089775:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.089786:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.089937:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74285056, count: 4096 00000080:00200000:0.0:1789760790.089946:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74285056, bytes: 4096 00000080:00200000:0.0:1789760790.089959:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.089968:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.089971:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74285056, 74289152) 0 0 00000080:00200000:0.0:1789760790.089975:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.089994:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18136, 18136] 00020000:00200000:0.0:1789760790.090002:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74285056 chunk: [74285056, 74289152) 74289152, 4096 00020000:00200000:0.0:1789760790.090007:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.090025:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18136, 18136] 00000008:00010000:0.0:1789760790.090042:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.090048:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.090061:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74285056, 74289152) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.090074:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74285056=0x46d8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.090135:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.090138:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74289152 00000020:00010000:0.0:1789760790.090147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.090150:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.090153:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.090157:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.090160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.090168:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.090176:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.090295:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760790.090435:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 74285056<>74289152 00000100:00100000:1.0:1789760790.090499:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565552256:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.104576:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985444 gen 3 00000008:00010000:1.0:1789760790.104635:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760790.105117:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.105130:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.105150:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:1.0:1789760790.105182:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565552256:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.105291:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18136, 18136] 00000080:00200000:0.0:1789760790.105309:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.105315:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.105329:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74289152 i_blocks 145120 00000020:00200000:0.0:1789760790.105332:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.105344:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.105376:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74289152, count: 4096 00000080:00200000:0.0:1789760790.105384:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74289152, bytes: 4096 00000080:00200000:0.0:1789760790.105395:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.105405:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.105407:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74289152, 74293248) 0 0 00000080:00200000:0.0:1789760790.105411:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.105439:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18137, 18137] 00020000:00200000:0.0:1789760790.105448:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74289152 chunk: [74289152, 74293248) 74293248, 4096 00020000:00200000:0.0:1789760790.105453:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.105474:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18137, 18137] 00000008:00010000:0.0:1789760790.105502:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.105509:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.105521:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74289152, 74293248) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.105535:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74289152=0x46d9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.105596:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.105599:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74293248 00000020:00010000:0.0:1789760790.105607:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.105610:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.105614:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.105617:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.105620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.105628:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.105635:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.105681:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dcc0 00000008:00100000:0.0:1789760790.105998:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 74289152<>74293248 00000100:00100000:1.0:1789760790.106076:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565552384:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.116324:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985445 gen 3 00000008:00010000:1.0:1789760790.116392:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760790.116401:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.116413:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.116438:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760790.116468:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565552384:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.116581:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18137, 18137] 00000080:00200000:0.0:1789760790.116599:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.116605:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.116619:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74293248 i_blocks 145128 00000020:00200000:0.0:1789760790.116624:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.116635:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.116714:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74293248, count: 4096 00000080:00200000:0.0:1789760790.116722:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74293248, bytes: 4096 00000080:00200000:0.0:1789760790.116733:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.116741:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.116744:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74293248, 74297344) 0 0 00000080:00200000:0.0:1789760790.116748:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.116765:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18138, 18138] 00020000:00200000:0.0:1789760790.116773:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74293248 chunk: [74293248, 74297344) 74297344, 4096 00020000:00200000:0.0:1789760790.116777:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.116799:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18138, 18138] 00000008:00010000:0.0:1789760790.116823:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.116830:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.116842:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74293248, 74297344) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.116856:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74293248=0x46da000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.116916:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.116919:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74297344 00000020:00010000:0.0:1789760790.116928:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.116932:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.116935:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.116939:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.116941:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.116949:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.116956:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.116972:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760790.117095:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 74293248<>74297344 00000100:00100000:1.0:1789760790.117957:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760790.117985:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565552512:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.131279:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985446 gen 3 00000008:00010000:1.0:1789760790.131331:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760790.131339:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.131352:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.131384:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760790.131428:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565552512:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.131764:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18138, 18138] 00000080:00200000:0.0:1789760790.131789:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.131795:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.131809:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74297344 i_blocks 145136 00000020:00200000:0.0:1789760790.131813:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.131829:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.131873:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74297344, count: 4096 00000080:00200000:0.0:1789760790.131881:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74297344, bytes: 4096 00000080:00200000:0.0:1789760790.131892:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.131900:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.131903:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74297344, 74301440) 0 0 00000080:00200000:0.0:1789760790.131908:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.131924:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18139, 18139] 00020000:00200000:0.0:1789760790.131932:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74297344 chunk: [74297344, 74301440) 74301440, 4096 00020000:00200000:0.0:1789760790.131937:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.131957:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18139, 18139] 00000008:00010000:0.0:1789760790.131974:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.131981:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.131992:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74297344, 74301440) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.132006:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74297344=0x46db000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.132066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.132069:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74301440 00000020:00010000:0.0:1789760790.132078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.132082:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.132085:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.132089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.132092:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.132099:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.132107:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.132125:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760790.132253:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42680 - object 0x240000400:5 offset 74297344<>74301440 00000100:00100000:1.0:1789760790.132981:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565552640:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.142039:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985447 gen 3 00000008:00010000:1.0:1789760790.142092:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760790.142100:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.142110:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.142171:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760790.142201:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565552640:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.142283:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18139, 18139] 00000080:00200000:0.0:1789760790.142301:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.142308:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.142323:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74301440 i_blocks 145144 00000020:00200000:0.0:1789760790.142326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.142337:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.142446:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74301440, count: 4096 00000080:00200000:0.0:1789760790.142455:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74301440, bytes: 4096 00000080:00200000:0.0:1789760790.142466:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.142475:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.142478:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74301440, 74305536) 0 0 00000080:00200000:0.0:1789760790.142482:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.142559:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18140, 18140] 00020000:00200000:0.0:1789760790.142568:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74301440 chunk: [74301440, 74305536) 74305536, 4096 00020000:00200000:0.0:1789760790.142573:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.142594:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18140, 18140] 00000008:00010000:0.0:1789760790.142614:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.142621:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.142634:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74301440, 74305536) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.142742:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74301440=0x46dc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.142918:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.142921:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74305536 00000020:00010000:0.0:1789760790.142931:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.142934:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.142939:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.142942:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.142945:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.142954:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.142961:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.142983:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760790.143107:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 74301440<>74305536 00000100:00100000:1.0:1789760790.143181:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565552768:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.151184:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985448 gen 3 00000008:00010000:1.0:1789760790.151237:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760790.151246:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.151257:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.151280:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760790.151311:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565552768:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.152370:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18140, 18140] 00000080:00200000:0.0:1789760790.152394:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.152401:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.152418:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74305536 i_blocks 145152 00000020:00200000:0.0:1789760790.152433:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.152449:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.152859:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74305536, count: 4096 00000080:00200000:0.0:1789760790.152870:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74305536, bytes: 4096 00000080:00200000:0.0:1789760790.152885:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.152895:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.152898:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74305536, 74309632) 0 0 00000080:00200000:0.0:1789760790.152903:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.152920:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18141, 18141] 00020000:00200000:0.0:1789760790.152929:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74305536 chunk: [74305536, 74309632) 74309632, 4096 00020000:00200000:0.0:1789760790.152934:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.152957:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18141, 18141] 00000008:00010000:0.0:1789760790.152976:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.152983:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.152995:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74305536, 74309632) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.153009:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74305536=0x46dd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.153074:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.153077:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74309632 00000020:00010000:0.0:1789760790.153084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.153087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.153091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.153094:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.153097:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.153106:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.153113:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.153136:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760790.153269:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40700 - object 0x240000400:5 offset 74305536<>74309632 00000100:00100000:1.0:1789760790.153438:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565552896:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.188212:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985449 gen 3 00000008:00010000:1.0:1789760790.188263:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760790.188272:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.188283:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.188306:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:1.0:1789760790.188337:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565552896:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.188752:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18141, 18141] 00000080:00200000:0.0:1789760790.188780:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.188786:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.188800:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74309632 i_blocks 145160 00000020:00200000:0.0:1789760790.188804:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.188815:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.188846:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74309632, count: 4096 00000080:00200000:0.0:1789760790.188853:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74309632, bytes: 4096 00000080:00200000:0.0:1789760790.188862:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.188870:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.188873:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74309632, 74313728) 0 0 00000080:00200000:0.0:1789760790.188877:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.188895:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18142, 18142] 00020000:00200000:0.0:1789760790.188903:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74309632 chunk: [74309632, 74313728) 74313728, 4096 00020000:00200000:0.0:1789760790.188908:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.188925:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18142, 18142] 00000008:00010000:0.0:1789760790.188942:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.188948:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.188958:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74309632, 74313728) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.188970:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74309632=0x46de000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.189021:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.189023:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74313728 00000020:00010000:0.0:1789760790.189030:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.189034:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.189036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.189039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.189041:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.189047:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.189053:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.189068:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d300 00000008:00100000:0.0:1789760790.189184:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43100 - object 0x240000400:5 offset 74309632<>74313728 00000100:00100000:1.0:1789760790.191349:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760790.191391:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565553024:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.208379:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985450 gen 3 00000008:00010000:1.0:1789760790.208443:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760790.208451:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.208463:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.208688:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18142, 18142] 00000080:00200000:0.0:1789760790.208710:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.208716:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.208731:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74313728 i_blocks 145168 00000020:00200000:0.0:1789760790.208735:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.208748:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.208783:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74313728, count: 4096 00000080:00200000:0.0:1789760790.208792:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74313728, bytes: 4096 00000080:00200000:0.0:1789760790.208804:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.208813:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.208816:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74313728, 74317824) 0 0 00000080:00200000:0.0:1789760790.208821:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.208840:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18143, 18143] 00020000:00200000:0.0:1789760790.208848:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74313728 chunk: [74313728, 74317824) 74317824, 4096 00020000:00200000:0.0:1789760790.208854:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.208874:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18143, 18143] 00000008:00010000:0.0:1789760790.208895:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.208901:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.208914:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74313728, 74317824) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.208930:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74313728=0x46df000, pages 1 (max 131072), lockless, parallel 00000008:00010000:1.0:1789760790.209017:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760790.209050:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565553024:192.168.203.147@tcp:4:dd.0 00000020:00200000:0.0:1789760790.209725:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.209731:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74317824 00000020:00010000:0.0:1789760790.209742:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.209746:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.209750:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.209754:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.209757:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.209767:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.209775:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.209799:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760790.209941:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41f80 - object 0x240000400:5 offset 74313728<>74317824 00000100:00100000:1.0:1789760790.210023:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565553152:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.224807:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985451 gen 3 00000008:00010000:1.0:1789760790.224863:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760790.224871:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.224882:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.224908:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:1.0:1789760790.224943:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565553152:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.225104:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18143, 18143] 00000080:00200000:0.0:1789760790.225123:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.225129:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.225146:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74317824 i_blocks 145176 00000020:00200000:0.0:1789760790.225150:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.225163:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.225198:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74317824, count: 4096 00000080:00200000:0.0:1789760790.225206:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74317824, bytes: 4096 00000080:00200000:0.0:1789760790.225220:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.225229:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.225232:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74317824, 74321920) 0 0 00000080:00200000:0.0:1789760790.225237:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.225255:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18144, 18144] 00020000:00200000:0.0:1789760790.225265:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74317824 chunk: [74317824, 74321920) 74321920, 4096 00020000:00200000:0.0:1789760790.225269:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.225291:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18144, 18144] 00000008:00010000:0.0:1789760790.225309:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.225316:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.225328:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74317824, 74321920) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.225343:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74317824=0x46e0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.225552:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.225556:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74321920 00000020:00010000:0.0:1789760790.225567:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.225570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.225574:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.225578:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.225580:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.225588:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.225595:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.225768:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:1.0:1789760790.225907:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74317824<>74321920 00000100:00100000:1.0:1789760790.225943:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760790.225960:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565553280:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.243340:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985452 gen 3 00000008:00010000:1.0:1789760790.243405:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760790.243414:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.243704:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.243718:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760790.243750:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565553280:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.244239:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18144, 18144] 00000080:00200000:0.0:1789760790.244260:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.244266:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.244282:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74321920 i_blocks 145184 00000020:00200000:0.0:1789760790.244286:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.244297:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.244331:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74321920, count: 4096 00000080:00200000:0.0:1789760790.244338:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74321920, bytes: 4096 00000080:00200000:0.0:1789760790.244351:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.244757:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.244760:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74321920, 74326016) 0 0 00000080:00200000:0.0:1789760790.244823:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.244843:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18145, 18145] 00020000:00200000:0.0:1789760790.244853:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74321920 chunk: [74321920, 74326016) 74326016, 4096 00020000:00200000:0.0:1789760790.244857:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.244878:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18145, 18145] 00000008:00010000:0.0:1789760790.244898:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.244905:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.244917:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74321920, 74326016) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.244932:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74321920=0x46e1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.245062:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.245065:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74326016 00000020:00010000:0.0:1789760790.245075:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.245078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.245082:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.245086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.245089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.245097:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.245104:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.245124:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760790.245258:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 74321920<>74326016 00000100:00100000:0.0:1789760790.245300:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760790.245322:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565553408:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.255372:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985453 gen 3 00000008:00010000:1.0:1789760790.255438:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760790.255447:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.255458:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.255482:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760790.255511:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565553408:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.255626:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18145, 18145] 00000080:00200000:2.0:1789760790.255673:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.255679:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.255693:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74326016 i_blocks 145192 00000020:00200000:2.0:1789760790.255697:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.255708:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.255743:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74326016, count: 4096 00000080:00200000:2.0:1789760790.255751:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74326016, bytes: 4096 00000080:00200000:2.0:1789760790.255761:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.255771:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.255774:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74326016, 74330112) 0 0 00000080:00200000:2.0:1789760790.255778:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.255796:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18146, 18146] 00020000:00200000:2.0:1789760790.255804:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74326016 chunk: [74326016, 74330112) 74330112, 4096 00020000:00200000:2.0:1789760790.255809:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.255829:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18146, 18146] 00000008:00010000:2.0:1789760790.255847:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.255853:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.255864:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74326016, 74330112) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.255879:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74326016=0x46e2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.255937:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.255940:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74330112 00000020:00010000:2.0:1789760790.255948:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.255951:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.255954:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.255958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.255960:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.255968:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.255975:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.256006:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760790.256133:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74326016<>74330112 00000100:00100000:3.0:1789760790.259351:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760790.259518:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565553536:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.276155:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985454 gen 3 00000008:00010000:3.0:1789760790.276213:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760790.276222:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.276235:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.276327:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760790.276362:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565553536:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.276496:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18146, 18146] 00000080:00200000:2.0:1789760790.276513:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.276518:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.276693:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74330112 i_blocks 145200 00000020:00200000:2.0:1789760790.276699:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.276709:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.276812:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74330112, count: 4096 00000080:00200000:2.0:1789760790.276820:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74330112, bytes: 4096 00000080:00200000:2.0:1789760790.276831:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.276840:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.276842:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74330112, 74334208) 0 0 00000080:00200000:2.0:1789760790.276847:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.276865:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18147, 18147] 00020000:00200000:2.0:1789760790.276874:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74330112 chunk: [74330112, 74334208) 74334208, 4096 00020000:00200000:2.0:1789760790.276879:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.276900:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18147, 18147] 00000008:00010000:2.0:1789760790.276919:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.276926:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.276937:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74330112, 74334208) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.276950:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74330112=0x46e3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.277007:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.277010:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74334208 00000020:00010000:2.0:1789760790.277018:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.277021:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.277024:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.277027:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.277030:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.277038:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.277044:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.277062:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760790.277233:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 74330112<>74334208 00000100:00100000:3.0:1789760790.277296:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565553664:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.294766:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985455 gen 3 00000008:00010000:2.0:1789760790.294865:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760790.294874:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.294886:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.294916:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760790.294949:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565553664:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.295124:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18147, 18147] 00000080:00200000:1.0:1789760790.295143:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.295148:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.295163:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74334208 i_blocks 145208 00000020:00200000:1.0:1789760790.295168:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.295413:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.295459:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74334208, count: 4096 00000080:00200000:1.0:1789760790.295467:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74334208, bytes: 4096 00000080:00200000:1.0:1789760790.295480:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.295488:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.295491:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74334208, 74338304) 0 0 00000080:00200000:1.0:1789760790.295495:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.295510:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18148, 18148] 00020000:00200000:1.0:1789760790.295518:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74334208 chunk: [74334208, 74338304) 74338304, 4096 00020000:00200000:1.0:1789760790.296058:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.296087:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18148, 18148] 00000008:00010000:1.0:1789760790.296109:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.296116:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.296131:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74334208, 74338304) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.296147:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74334208=0x46e4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.296335:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.296339:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74338304 00000020:00010000:1.0:1789760790.296349:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.296353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.296499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.296504:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.296508:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.296518:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.296703:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.296733:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760790.296972:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74334208<>74338304 00000100:00100000:0.0:1789760790.297081:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760790.297293:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565553792:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.309394:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985456 gen 3 00000008:00010000:1.0:1789760790.309455:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:1.0:1789760790.309464:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.309485:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.309510:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:1.0:1789760790.309538:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565553792:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.309796:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18148, 18148] 00000080:00200000:2.0:1789760790.309812:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.309819:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.309834:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74338304 i_blocks 145216 00000020:00200000:2.0:1789760790.309839:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.309848:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.309884:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74338304, count: 4096 00000080:00200000:2.0:1789760790.309892:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74338304, bytes: 4096 00000080:00200000:2.0:1789760790.309902:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.309912:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.309914:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74338304, 74342400) 0 0 00000080:00200000:2.0:1789760790.309919:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.309936:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18149, 18149] 00020000:00200000:2.0:1789760790.309946:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74338304 chunk: [74338304, 74342400) 74342400, 4096 00020000:00200000:2.0:1789760790.309963:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.309985:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18149, 18149] 00000008:00010000:2.0:1789760790.310005:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.310011:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.310023:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74338304, 74342400) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.310038:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74338304=0x46e5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.310096:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.310099:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74342400 00000020:00010000:2.0:1789760790.310107:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.310111:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.310114:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.310117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.310120:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.310127:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.310135:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.310150:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760790.310272:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 74338304<>74342400 00000100:00100000:3.0:1789760790.310353:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565553920:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.336780:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985457 gen 3 00000008:00010000:2.0:1789760790.336831:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760790.336840:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.336850:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.336874:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760790.336901:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565553920:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.337054:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18149, 18149] 00000080:00200000:1.0:1789760790.337070:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.337076:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.337092:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74342400 i_blocks 145224 00000020:00200000:1.0:1789760790.337096:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.337105:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.337138:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74342400, count: 4096 00000080:00200000:1.0:1789760790.337146:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74342400, bytes: 4096 00000080:00200000:1.0:1789760790.337156:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.337164:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.337167:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74342400, 74346496) 0 0 00000080:00200000:1.0:1789760790.337171:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.337189:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18150, 18150] 00020000:00200000:1.0:1789760790.337198:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74342400 chunk: [74342400, 74346496) 74346496, 4096 00020000:00200000:1.0:1789760790.337203:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.337224:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18150, 18150] 00000008:00010000:1.0:1789760790.337244:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.337250:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.337263:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74342400, 74346496) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.337277:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74342400=0x46e6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.337335:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.337337:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74346496 00000020:00010000:1.0:1789760790.337345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.337349:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.337351:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.337355:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.337357:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.337364:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.337371:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.337388:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760790.337525:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74342400<>74346496 00000100:00100000:1.0:1789760790.337581:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565554048:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.354150:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985458 gen 3 00000008:00010000:1.0:1789760790.354204:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:1.0:1789760790.354213:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.354224:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.354253:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:1.0:1789760790.354284:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565554048:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.354429:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18150, 18150] 00000080:00200000:2.0:1789760790.354448:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.354455:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.354472:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74346496 i_blocks 145232 00000020:00200000:2.0:1789760790.354488:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.354498:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.354540:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74346496, count: 4096 00000080:00200000:2.0:1789760790.354548:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74346496, bytes: 4096 00000080:00200000:2.0:1789760790.354560:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.354568:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.354571:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74346496, 74350592) 0 0 00000080:00200000:2.0:1789760790.354577:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.354596:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18151, 18151] 00020000:00200000:2.0:1789760790.354605:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74346496 chunk: [74346496, 74350592) 74350592, 4096 00020000:00200000:2.0:1789760790.354609:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.354630:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18151, 18151] 00000008:00010000:2.0:1789760790.354694:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.354700:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.354714:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74346496, 74350592) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.354741:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74346496=0x46e7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.354803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.354806:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74350592 00000020:00010000:2.0:1789760790.354813:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.354817:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.354820:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.354824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.354827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.354835:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.354842:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.354859:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760790.354985:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 74346496<>74350592 00000100:00100000:2.0:1789760790.355037:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760790.355055:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565554176:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.375884:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985459 gen 3 00000008:00010000:2.0:1789760790.375935:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760790.375942:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.375954:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.375969:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760790.375997:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565554176:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.381582:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18151, 18151] 00000080:00200000:2.0:1789760790.381610:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.381617:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.381636:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74350592 i_blocks 145240 00000020:00200000:2.0:1789760790.381641:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.381696:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.381731:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74350592, count: 4096 00000080:00200000:2.0:1789760790.381741:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74350592, bytes: 4096 00000080:00200000:2.0:1789760790.381755:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.381766:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.381768:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74350592, 74354688) 0 0 00000080:00200000:2.0:1789760790.381773:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.381790:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18152, 18152] 00020000:00200000:2.0:1789760790.381800:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74350592 chunk: [74350592, 74354688) 74354688, 4096 00020000:00200000:2.0:1789760790.381805:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.381827:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18152, 18152] 00000008:00010000:2.0:1789760790.381846:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.381852:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.381864:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74350592, 74354688) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.381880:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74350592=0x46e8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.381945:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.381948:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74354688 00000020:00010000:2.0:1789760790.381956:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.381960:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.381964:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.381968:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.381971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.381979:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.381986:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.382007:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760790.382200:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 74350592<>74354688 00000100:00100000:3.0:1789760790.382435:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565554304:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.400522:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985460 gen 3 00000008:00010000:3.0:1789760790.400573:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760790.400582:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.400593:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.400620:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760790.400692:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565554304:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.400947:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18152, 18152] 00000080:00200000:2.0:1789760790.400966:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.400973:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.400990:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74354688 i_blocks 145248 00000020:00200000:2.0:1789760790.400994:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.401004:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.401039:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74354688, count: 4096 00000080:00200000:2.0:1789760790.401047:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74354688, bytes: 4096 00000080:00200000:2.0:1789760790.401058:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.401066:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.401069:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74354688, 74358784) 0 0 00000080:00200000:2.0:1789760790.401074:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.401090:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18153, 18153] 00020000:00200000:2.0:1789760790.401099:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74354688 chunk: [74354688, 74358784) 74358784, 4096 00020000:00200000:2.0:1789760790.401103:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.401123:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18153, 18153] 00000008:00010000:2.0:1789760790.401142:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.401148:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.401237:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74354688, 74358784) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.401251:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74354688=0x46e9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.401315:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.401318:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74358784 00000020:00010000:2.0:1789760790.401327:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.401330:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.401335:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.401339:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.401342:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.401350:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.401417:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.401445:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760790.401568:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 74354688<>74358784 00000100:00100000:3.0:1789760790.401673:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565554432:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.413297:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985461 gen 3 00000008:00010000:3.0:1789760790.413350:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760790.413369:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.413380:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.413403:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760790.413444:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565554432:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.413606:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18153, 18153] 00000080:00200000:2.0:1789760790.413624:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.413631:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.413680:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74358784 i_blocks 145256 00000020:00200000:2.0:1789760790.413684:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.413694:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.413727:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74358784, count: 4096 00000080:00200000:2.0:1789760790.413734:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74358784, bytes: 4096 00000080:00200000:2.0:1789760790.413745:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.413753:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.413756:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74358784, 74362880) 0 0 00000080:00200000:2.0:1789760790.413761:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.413779:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18154, 18154] 00020000:00200000:2.0:1789760790.413788:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74358784 chunk: [74358784, 74362880) 74362880, 4096 00020000:00200000:2.0:1789760790.413793:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.413814:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18154, 18154] 00000008:00010000:2.0:1789760790.413832:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.413839:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.413851:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74358784, 74362880) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.413865:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74358784=0x46ea000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.413923:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.413926:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74362880 00000020:00010000:2.0:1789760790.413934:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.413938:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.413942:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.413945:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.413947:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.413955:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.413963:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.413980:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760790.414101:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 74358784<>74362880 00000100:00100000:3.0:1789760790.414699:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565554560:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.422447:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985462 gen 3 00000008:00010000:3.0:1789760790.422508:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760790.422517:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.422529:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.422551:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760790.422583:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565554560:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.422799:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18154, 18154] 00000080:00200000:2.0:1789760790.422818:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.422824:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.422839:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74362880 i_blocks 145264 00000020:00200000:2.0:1789760790.422843:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.422853:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.422888:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74362880, count: 4096 00000080:00200000:2.0:1789760790.422896:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74362880, bytes: 4096 00000080:00200000:2.0:1789760790.422906:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.422915:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.422918:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74362880, 74366976) 0 0 00000080:00200000:2.0:1789760790.422922:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.422941:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18155, 18155] 00020000:00200000:2.0:1789760790.422949:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74362880 chunk: [74362880, 74366976) 74366976, 4096 00020000:00200000:2.0:1789760790.422954:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.422974:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18155, 18155] 00000008:00010000:2.0:1789760790.422992:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.422998:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.423010:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74362880, 74366976) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.423024:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74362880=0x46eb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.423082:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.423085:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74366976 00000020:00010000:2.0:1789760790.423093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.423096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.423100:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.423104:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.423107:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.423115:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.423122:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.423138:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760790.423870:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 74362880<>74366976 00000100:00100000:3.0:1789760790.424071:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760790.424094:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565554688:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.438586:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985463 gen 3 00000008:00010000:3.0:1789760790.438639:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760790.438686:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.438697:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.438723:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760790.438759:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565554688:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.438878:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18155, 18155] 00000080:00200000:2.0:1789760790.438895:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.438901:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.438915:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74366976 i_blocks 145272 00000020:00200000:2.0:1789760790.438920:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.438931:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.438963:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74366976, count: 4096 00000080:00200000:2.0:1789760790.438971:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74366976, bytes: 4096 00000080:00200000:2.0:1789760790.438982:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.438988:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.438990:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74366976, 74371072) 0 0 00000080:00200000:2.0:1789760790.438994:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.439009:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18156, 18156] 00020000:00200000:2.0:1789760790.439016:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74366976 chunk: [74366976, 74371072) 74371072, 4096 00020000:00200000:2.0:1789760790.439020:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.439037:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18156, 18156] 00000008:00010000:2.0:1789760790.439050:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.439055:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.439065:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74366976, 74371072) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.439076:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74366976=0x46ec000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.439121:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.439123:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74371072 00000020:00010000:2.0:1789760790.439129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.439131:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.439134:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.439136:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.439138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.439144:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.439149:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.439223:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760790.439348:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 74366976<>74371072 00000100:00100000:3.0:1789760790.439528:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760790.439546:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565554816:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.450143:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985464 gen 3 00000008:00010000:3.0:1789760790.450196:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760790.450205:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.450218:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.450301:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760790.450335:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565554816:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.450682:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18156, 18156] 00000080:00200000:2.0:1789760790.450700:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.450707:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.450723:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74371072 i_blocks 145280 00000020:00200000:2.0:1789760790.450727:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.450738:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.450774:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74371072, count: 4096 00000080:00200000:2.0:1789760790.450781:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74371072, bytes: 4096 00000080:00200000:2.0:1789760790.450792:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.450801:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.450803:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74371072, 74375168) 0 0 00000080:00200000:2.0:1789760790.450808:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.450826:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18157, 18157] 00020000:00200000:2.0:1789760790.450835:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74371072 chunk: [74371072, 74375168) 74375168, 4096 00020000:00200000:2.0:1789760790.450840:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.450860:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18157, 18157] 00000008:00010000:2.0:1789760790.450878:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.450884:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.450898:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74371072, 74375168) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.450911:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74371072=0x46ed000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.450973:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.450975:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74375168 00000020:00010000:2.0:1789760790.450984:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.450988:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.450992:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.450996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.450998:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.451007:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.451014:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.451034:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760790.451162:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 74371072<>74375168 00000100:00100000:3.0:1789760790.451242:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565554944:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.459985:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985465 gen 3 00000008:00010000:2.0:1789760790.460037:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760790.460045:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.460057:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.460112:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760790.460138:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565554944:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.460217:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18157, 18157] 00000080:00200000:0.0:1789760790.460235:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.460242:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.460345:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74375168 i_blocks 145288 00000020:00200000:2.0:1789760790.460349:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.460357:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.460387:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74375168, count: 4096 00000080:00200000:2.0:1789760790.460394:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74375168, bytes: 4096 00000080:00200000:2.0:1789760790.460410:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.460429:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.460432:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74375168, 74379264) 0 0 00000080:00200000:2.0:1789760790.460437:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.460454:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18158, 18158] 00020000:00200000:2.0:1789760790.460463:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74375168 chunk: [74375168, 74379264) 74379264, 4096 00020000:00200000:2.0:1789760790.460468:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.460549:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18158, 18158] 00000008:00010000:2.0:1789760790.460569:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.460575:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.460589:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74375168, 74379264) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.460605:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74375168=0x46ee000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.460678:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.460680:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74379264 00000020:00010000:2.0:1789760790.460688:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.460690:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.460692:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.460695:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.460697:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.460703:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.460708:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.460862:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760790.460977:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 74375168<>74379264 00000100:00100000:3.0:1789760790.461045:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565555072:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.471081:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985466 gen 3 00000008:00010000:2.0:1789760790.471390:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760790.471399:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.471411:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.471493:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760790.471572:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565555072:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.471835:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18158, 18158] 00000080:00200000:0.0:1789760790.471852:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.471858:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.471874:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74379264 i_blocks 145296 00000020:00200000:0.0:1789760790.471878:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.471888:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.471923:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74379264, count: 4096 00000080:00200000:0.0:1789760790.471930:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74379264, bytes: 4096 00000080:00200000:0.0:1789760790.471942:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.471951:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.471954:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74379264, 74383360) 0 0 00000080:00200000:0.0:1789760790.471959:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.471978:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18159, 18159] 00020000:00200000:0.0:1789760790.471987:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74379264 chunk: [74379264, 74383360) 74383360, 4096 00020000:00200000:0.0:1789760790.471991:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.472012:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18159, 18159] 00000008:00010000:0.0:1789760790.472032:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.472038:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.472050:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74379264, 74383360) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.472064:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74379264=0x46ef000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.472124:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.472812:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74383360 00000020:00010000:0.0:1789760790.472824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.472828:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.472832:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.472835:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.472840:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.472849:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.472857:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.472884:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760790.473278:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 74379264<>74383360 00000100:00100000:1.0:1789760790.473368:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565555200:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.481139:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985467 gen 3 00000008:00010000:1.0:1789760790.481188:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760790.481196:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.481207:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.481376:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18159, 18159] 00000080:00200000:0.0:1789760790.481396:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.481402:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.481429:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74383360 i_blocks 145304 00000020:00200000:0.0:1789760790.481434:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.481447:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.481483:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74383360, count: 4096 00000080:00200000:0.0:1789760790.481491:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74383360, bytes: 4096 00000080:00200000:0.0:1789760790.481502:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.481511:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.481514:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74383360, 74387456) 0 0 00000080:00200000:0.0:1789760790.481518:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.481536:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18160, 18160] 00020000:00200000:0.0:1789760790.481544:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74383360 chunk: [74383360, 74387456) 74387456, 4096 00020000:00200000:0.0:1789760790.481549:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.481570:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18160, 18160] 00000008:00010000:0.0:1789760790.481590:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.481596:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.481609:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74383360, 74387456) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.481623:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74383360=0x46f0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.481748:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.481751:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74387456 00000020:00010000:0.0:1789760790.481760:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.481763:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.481767:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.481770:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.481774:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.481781:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.481789:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.481810:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760790.481935:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 74383360<>74387456 00000100:00100000:0.0:1789760790.481988:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565555328:192.168.203.147@tcp:4:dd.0 00000008:00010000:1.0:1789760790.482502:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:1.0:1789760790.482528:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565555200:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.494366:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985468 gen 3 00000008:00010000:1.0:1789760790.494429:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760790.494438:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.494451:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.494471:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760790.494500:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565555328:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.494754:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18160, 18160] 00000080:00200000:0.0:1789760790.494772:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.494777:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.494793:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74387456 i_blocks 145312 00000020:00200000:0.0:1789760790.494797:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.494808:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.494843:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74387456, count: 4096 00000080:00200000:0.0:1789760790.494851:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74387456, bytes: 4096 00000080:00200000:0.0:1789760790.494863:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.494872:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.494874:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74387456, 74391552) 0 0 00000080:00200000:0.0:1789760790.494879:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.494897:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18161, 18161] 00020000:00200000:0.0:1789760790.494905:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74387456 chunk: [74387456, 74391552) 74391552, 4096 00020000:00200000:0.0:1789760790.494910:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.494930:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18161, 18161] 00000008:00010000:0.0:1789760790.494947:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.494953:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.494966:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74387456, 74391552) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.494980:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74387456=0x46f1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.495035:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.495039:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74391552 00000020:00010000:0.0:1789760790.495047:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.495050:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.495054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.495057:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.495059:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.495065:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.495072:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.495089:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760790.495213:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 74387456<>74391552 00000100:00100000:1.0:1789760790.496578:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760790.496611:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565555456:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.508431:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985469 gen 3 00000008:00010000:1.0:1789760790.508495:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760790.508504:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.508514:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.508534:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:1.0:1789760790.508564:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565555456:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.508946:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18161, 18161] 00000080:00200000:0.0:1789760790.508963:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.508970:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.508986:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74391552 i_blocks 145320 00000020:00200000:0.0:1789760790.508990:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.509001:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.509037:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74391552, count: 4096 00000080:00200000:0.0:1789760790.509044:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74391552, bytes: 4096 00000080:00200000:0.0:1789760790.509054:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.509063:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.509065:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74391552, 74395648) 0 0 00000080:00200000:0.0:1789760790.509069:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.509086:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18162, 18162] 00020000:00200000:0.0:1789760790.509094:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74391552 chunk: [74391552, 74395648) 74395648, 4096 00020000:00200000:0.0:1789760790.509099:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.509119:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18162, 18162] 00000008:00010000:0.0:1789760790.509136:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.509142:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.509154:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74391552, 74395648) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.509168:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74391552=0x46f2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.509227:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.509230:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74395648 00000020:00010000:0.0:1789760790.509239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.509342:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.509347:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.509351:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.509355:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.509364:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.509371:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.509389:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:0.0:1789760790.510003:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 74391552<>74395648 00000100:00100000:1.0:1789760790.510072:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565555584:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.524767:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985470 gen 3 00000008:00010000:1.0:1789760790.524818:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760790.524828:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.524839:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.524869:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760790.524899:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565555584:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.525088:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18162, 18162] 00000080:00200000:2.0:1789760790.525106:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.525112:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.525128:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74395648 i_blocks 145328 00000020:00200000:2.0:1789760790.525132:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.525144:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.525183:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74395648, count: 4096 00000080:00200000:2.0:1789760790.525191:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74395648, bytes: 4096 00000080:00200000:2.0:1789760790.525203:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.525212:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.525215:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74395648, 74399744) 0 0 00000080:00200000:2.0:1789760790.525219:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.525237:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18163, 18163] 00020000:00200000:2.0:1789760790.525247:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74395648 chunk: [74395648, 74399744) 74399744, 4096 00020000:00200000:2.0:1789760790.525252:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.525273:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18163, 18163] 00000008:00010000:2.0:1789760790.525291:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.525298:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.525310:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74395648, 74399744) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.525324:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74395648=0x46f3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.525573:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.525577:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74399744 00000020:00010000:2.0:1789760790.525587:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.525590:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.525593:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.525597:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.525600:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.525608:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.525616:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.525633:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760790.525787:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 74395648<>74399744 00000100:00100000:2.0:1789760790.525834:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760790.525852:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565555712:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.534455:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985471 gen 3 00000008:00010000:2.0:1789760790.534507:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760790.534516:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.534526:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.534552:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760790.534581:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565555712:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.535191:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18163, 18163] 00000080:00200000:1.0:1789760790.535210:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.535216:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.535233:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74399744 i_blocks 145336 00000020:00200000:1.0:1789760790.535237:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.535248:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.535282:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74399744, count: 4096 00000080:00200000:1.0:1789760790.535290:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74399744, bytes: 4096 00000080:00200000:1.0:1789760790.535300:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.535308:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.535311:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74399744, 74403840) 0 0 00000080:00200000:1.0:1789760790.535315:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.535332:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18164, 18164] 00020000:00200000:1.0:1789760790.535340:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74399744 chunk: [74399744, 74403840) 74403840, 4096 00020000:00200000:1.0:1789760790.535344:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.535699:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18164, 18164] 00000008:00010000:1.0:1789760790.535719:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.535724:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.535736:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74399744, 74403840) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.535749:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74399744=0x46f4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.535798:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.535801:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74403840 00000020:00010000:1.0:1789760790.535809:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.535813:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.535815:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.535819:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.535822:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.535829:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.535835:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.538207:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:2.0:1789760790.538365:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 74399744<>74403840 00000100:00100000:3.0:1789760790.538487:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760790.541634:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565555840:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.557062:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985472 gen 3 00000008:00010000:2.0:1789760790.557112:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:2.0:1789760790.557120:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.557131:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.557153:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:2.0:1789760790.557184:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565555840:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.557343:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18164, 18164] 00000080:00200000:1.0:1789760790.557360:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.557366:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.557383:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74403840 i_blocks 145344 00000020:00200000:1.0:1789760790.557387:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.557399:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.557444:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74403840, count: 4096 00000080:00200000:1.0:1789760790.557451:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74403840, bytes: 4096 00000080:00200000:1.0:1789760790.557461:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.557470:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.557472:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74403840, 74407936) 0 0 00000080:00200000:1.0:1789760790.557477:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.557496:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18165, 18165] 00020000:00200000:1.0:1789760790.557505:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74403840 chunk: [74403840, 74407936) 74407936, 4096 00020000:00200000:1.0:1789760790.557510:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.557530:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18165, 18165] 00000008:00010000:1.0:1789760790.557547:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.557553:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.557565:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74403840, 74407936) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.557580:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74403840=0x46f5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.557680:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.557683:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74407936 00000020:00010000:1.0:1789760790.557692:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.557696:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.557699:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.557703:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.557706:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.557714:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.557722:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.557740:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760790.557885:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74403840<>74407936 00000100:00100000:1.0:1789760790.557926:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760790.557943:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565555968:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.571347:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985473 gen 3 00000008:00010000:1.0:1789760790.571895:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:1.0:1789760790.571905:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.571916:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.572243:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:1.0:1789760790.572277:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565555968:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.572399:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18165, 18165] 00000080:00200000:2.0:1789760790.572415:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.572431:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.572447:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74407936 i_blocks 145352 00000020:00200000:2.0:1789760790.572451:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.572462:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.572504:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74407936, count: 4096 00000080:00200000:2.0:1789760790.572512:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74407936, bytes: 4096 00000080:00200000:2.0:1789760790.572525:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.572533:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.572536:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74407936, 74412032) 0 0 00000080:00200000:2.0:1789760790.572540:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.572555:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18166, 18166] 00020000:00200000:2.0:1789760790.572566:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74407936 chunk: [74407936, 74412032) 74412032, 4096 00020000:00200000:2.0:1789760790.572571:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.572735:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18166, 18166] 00000008:00010000:2.0:1789760790.572754:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.572760:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.572785:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74407936, 74412032) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.572799:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74407936=0x46f6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.572858:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.572862:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74412032 00000020:00010000:2.0:1789760790.572871:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.572874:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.572877:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.572881:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.572884:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.572892:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.572900:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.572916:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760790.573034:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 74407936<>74412032 00000100:00100000:2.0:1789760790.573086:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760790.573103:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565556096:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.583874:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985474 gen 3 00000008:00010000:2.0:1789760790.583920:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760790.583928:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.583939:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.584111:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760790.584141:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565556096:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.584215:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18166, 18166] 00000080:00200000:1.0:1789760790.584231:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.584237:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.584253:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74412032 i_blocks 145360 00000020:00200000:1.0:1789760790.584257:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.584266:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.584297:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74412032, count: 4096 00000080:00200000:1.0:1789760790.584304:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74412032, bytes: 4096 00000080:00200000:1.0:1789760790.584314:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.584322:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.584325:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74412032, 74416128) 0 0 00000080:00200000:1.0:1789760790.584329:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.584344:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18167, 18167] 00020000:00200000:1.0:1789760790.584352:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74412032 chunk: [74412032, 74416128) 74416128, 4096 00020000:00200000:1.0:1789760790.584356:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.584374:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18167, 18167] 00000008:00010000:1.0:1789760790.584391:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.584396:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.584407:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74412032, 74416128) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.584428:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74412032=0x46f7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.584481:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.584483:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74416128 00000020:00010000:1.0:1789760790.584554:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.584558:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.584561:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.584565:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.584569:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.584576:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.584582:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.584599:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760790.584736:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74412032<>74416128 00000100:00100000:1.0:1789760790.584788:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565556224:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.593363:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985475 gen 3 00000008:00010000:1.0:1789760790.593450:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:1.0:1789760790.593459:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.593471:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.593500:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:1.0:1789760790.593531:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565556224:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.593544:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18167, 18167] 00000080:00200000:2.0:1789760790.593562:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.593568:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.593582:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74416128 i_blocks 145368 00000020:00200000:2.0:1789760790.593595:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.593606:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.593640:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74416128, count: 4096 00000080:00200000:2.0:1789760790.593681:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74416128, bytes: 4096 00000080:00200000:2.0:1789760790.593692:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.593701:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.593703:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74416128, 74420224) 0 0 00000080:00200000:2.0:1789760790.593707:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.593725:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18168, 18168] 00020000:00200000:2.0:1789760790.593734:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74416128 chunk: [74416128, 74420224) 74420224, 4096 00020000:00200000:2.0:1789760790.593739:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.593758:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18168, 18168] 00000008:00010000:2.0:1789760790.593783:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.593789:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.593802:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74416128, 74420224) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.593817:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74416128=0x46f8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.593873:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.593876:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74420224 00000020:00010000:2.0:1789760790.593884:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.593887:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.593890:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.593894:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.593897:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.593905:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.593911:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.593924:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760790.594036:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 74416128<>74420224 00000100:00100000:3.0:1789760790.594141:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565556352:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.605333:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985476 gen 3 00000008:00010000:3.0:1789760790.605884:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:3.0:1789760790.605894:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.605905:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.605927:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:3.0:1789760790.605962:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565556352:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.606083:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18168, 18168] 00000080:00200000:2.0:1789760790.606101:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.606107:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.606124:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74420224 i_blocks 145376 00000020:00200000:2.0:1789760790.606128:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.606138:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.606243:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74420224, count: 4096 00000080:00200000:2.0:1789760790.606251:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74420224, bytes: 4096 00000080:00200000:2.0:1789760790.606262:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.606271:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.606274:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74420224, 74424320) 0 0 00000080:00200000:2.0:1789760790.606278:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.606296:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18169, 18169] 00020000:00200000:2.0:1789760790.606304:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74420224 chunk: [74420224, 74424320) 74424320, 4096 00020000:00200000:2.0:1789760790.606310:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.606330:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18169, 18169] 00000008:00010000:2.0:1789760790.606349:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.606720:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.606736:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74420224, 74424320) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.606751:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74420224=0x46f9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.606883:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.606887:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74424320 00000020:00010000:2.0:1789760790.606896:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.606900:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.606903:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.606906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.606909:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.606918:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.606924:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.606944:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760790.607070:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74420224<>74424320 00000100:00100000:3.0:1789760790.607254:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760790.607278:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565556480:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.623342:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985477 gen 3 00000008:00010000:2.0:1789760790.623394:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760790.623402:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.623476:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.623563:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000080:00200000:1.0:1789760790.623734:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18169, 18169] 00000080:00200000:1.0:1789760790.623753:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.623759:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.623775:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74424320 i_blocks 145384 00000020:00200000:1.0:1789760790.623779:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.623790:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.623825:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74424320, count: 4096 00000080:00200000:1.0:1789760790.623834:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74424320, bytes: 4096 00000080:00200000:1.0:1789760790.623847:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.623856:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.623859:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74424320, 74428416) 0 0 00000080:00200000:1.0:1789760790.623864:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.623883:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18170, 18170] 00020000:00200000:1.0:1789760790.623891:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74424320 chunk: [74424320, 74428416) 74428416, 4096 00020000:00200000:1.0:1789760790.623896:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.623915:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18170, 18170] 00000008:00010000:1.0:1789760790.623933:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.623939:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.623953:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74424320, 74428416) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.623967:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74424320=0x46fa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.624017:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.624020:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74428416 00000020:00010000:1.0:1789760790.624029:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.624033:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.624036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.624039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.624042:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.624049:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.624056:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:2.0:1789760790.624124:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565556480:192.168.203.147@tcp:4:dd.0 00000008:00010000:2.0:1789760790.624246:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:2.0:1789760790.624368:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74424320<>74428416 00000100:00100000:3.0:1789760790.624450:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760790.626868:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565556608:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.638328:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985478 gen 3 00000008:00010000:2.0:1789760790.638378:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:2.0:1789760790.638387:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.638398:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.638437:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:2.0:1789760790.638469:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565556608:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.638755:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18170, 18170] 00000080:00200000:1.0:1789760790.638844:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.638851:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.638867:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74428416 i_blocks 145392 00000020:00200000:1.0:1789760790.638871:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.638882:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.638917:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74428416, count: 4096 00000080:00200000:1.0:1789760790.638923:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74428416, bytes: 4096 00000080:00200000:1.0:1789760790.638935:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.638944:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.638947:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74428416, 74432512) 0 0 00000080:00200000:1.0:1789760790.638951:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.638968:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18171, 18171] 00020000:00200000:1.0:1789760790.638975:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74428416 chunk: [74428416, 74432512) 74432512, 4096 00020000:00200000:1.0:1789760790.638980:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.638998:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18171, 18171] 00000008:00010000:1.0:1789760790.639015:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.639021:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.639033:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74428416, 74432512) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.639046:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74428416=0x46fb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.639113:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.639116:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74432512 00000020:00010000:1.0:1789760790.639126:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.639129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.639133:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.639136:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.639139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.639901:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.639916:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.639955:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760790.640106:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74428416<>74432512 00000100:00100000:1.0:1789760790.640732:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760790.640772:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565556736:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.653800:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985479 gen 3 00000008:00010000:1.0:1789760790.653856:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760790.653866:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.653878:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.653905:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760790.653935:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565556736:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.654061:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18171, 18171] 00000080:00200000:2.0:1789760790.654078:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.654084:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.654100:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74432512 i_blocks 145400 00000020:00200000:2.0:1789760790.654103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.654114:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.654159:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74432512, count: 4096 00000080:00200000:2.0:1789760790.654167:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74432512, bytes: 4096 00000080:00200000:2.0:1789760790.654178:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.654187:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.654189:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74432512, 74436608) 0 0 00000080:00200000:2.0:1789760790.654194:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.654210:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18172, 18172] 00020000:00200000:2.0:1789760790.654220:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74432512 chunk: [74432512, 74436608) 74436608, 4096 00020000:00200000:2.0:1789760790.654225:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.654244:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18172, 18172] 00000008:00010000:2.0:1789760790.654262:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.654268:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.654279:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74432512, 74436608) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.654292:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74432512=0x46fc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.654352:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.654355:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74436608 00000020:00010000:2.0:1789760790.654363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.654368:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.654371:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.654375:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.654378:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.654386:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.654393:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.654409:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760790.654544:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74432512<>74436608 00000100:00100000:2.0:1789760790.654593:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760790.654610:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565556864:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.670905:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985480 gen 3 00000008:00010000:2.0:1789760790.670959:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760790.670968:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.670979:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.671074:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760790.671105:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565556864:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760790.671235:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18172, 18172] 00000080:00200000:3.0:1789760790.671251:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760790.671258:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.671964:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74436608 i_blocks 145408 00000020:00200000:2.0:1789760790.671971:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.671983:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.672019:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74436608, count: 4096 00000080:00200000:2.0:1789760790.672028:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74436608, bytes: 4096 00000080:00200000:2.0:1789760790.672043:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.672053:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.672055:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74436608, 74440704) 0 0 00000080:00200000:2.0:1789760790.672061:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.672079:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18173, 18173] 00020000:00200000:2.0:1789760790.672090:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74436608 chunk: [74436608, 74440704) 74440704, 4096 00020000:00200000:2.0:1789760790.672095:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.672117:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18173, 18173] 00000008:00010000:2.0:1789760790.672137:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.672144:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.672157:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74436608, 74440704) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.672174:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74436608=0x46fd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.672242:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.672245:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74440704 00000020:00010000:2.0:1789760790.672255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.672258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.672261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354ee18/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.672264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.672267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.672276:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.672283:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.672834:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760790.672955:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74436608<>74440704 00000100:00100000:2.0:1789760790.673008:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565556992:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.691102:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985481 gen 3 00000008:00010000:2.0:1789760790.691157:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760790.691166:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.691177:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.691264:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760790.691296:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565556992:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760790.692748:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18173, 18173] 00000080:00200000:3.0:1789760790.692882:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760790.692890:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.693049:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74440704 i_blocks 145416 00000020:00200000:2.0:1789760790.693055:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.693067:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.693101:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74440704, count: 4096 00000080:00200000:2.0:1789760790.693109:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74440704, bytes: 4096 00000080:00200000:2.0:1789760790.693125:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.693134:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.693137:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74440704, 74444800) 0 0 00000080:00200000:2.0:1789760790.693141:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.693159:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18174, 18174] 00020000:00200000:2.0:1789760790.693170:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74440704 chunk: [74440704, 74444800) 74444800, 4096 00020000:00200000:2.0:1789760790.693175:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.693197:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18174, 18174] 00000008:00010000:2.0:1789760790.693216:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.693223:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.693235:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74440704, 74444800) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.693249:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74440704=0x46fe000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.693312:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.693315:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74444800 00000020:00010000:2.0:1789760790.693323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.693326:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.693329:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e2d8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.693333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.693336:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.693344:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.693351:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.694480:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760790.694627:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74440704<>74444800 00000100:00100000:2.0:1789760790.694703:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760790.694727:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565557120:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.709268:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985482 gen 3 00000008:00010000:2.0:1789760790.709319:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760790.709327:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.709337:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.709367:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760790.709398:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565557120:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.709594:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18174, 18174] 00000080:00200000:1.0:1789760790.709612:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.709618:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.709634:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74444800 i_blocks 145424 00000020:00200000:1.0:1789760790.709638:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.709679:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.709715:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74444800, count: 4096 00000080:00200000:1.0:1789760790.709723:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74444800, bytes: 4096 00000080:00200000:1.0:1789760790.709737:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.709746:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.709748:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74444800, 74448896) 0 0 00000080:00200000:1.0:1789760790.709752:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.709768:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18175, 18175] 00020000:00200000:1.0:1789760790.709776:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74444800 chunk: [74444800, 74448896) 74448896, 4096 00020000:00200000:1.0:1789760790.709780:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.709798:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18175, 18175] 00000008:00010000:1.0:1789760790.709815:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.709820:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.709831:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74444800, 74448896) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.709844:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74444800=0x46ff000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.709898:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.709901:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74448896 00000020:00010000:1.0:1789760790.709909:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.709912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.709915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e2d8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.709932:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.709935:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.709942:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.709949:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.709968:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760790.710090:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74444800<>74448896 00000100:00100000:1.0:1789760790.710143:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565557248:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.723773:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985483 gen 3 00000008:00010000:1.0:1789760790.723826:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760790.723835:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.723847:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.725891:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760790.725943:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565557248:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.727536:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18175, 18175] 00000080:00200000:2.0:1789760790.727559:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.727566:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.727586:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74448896 i_blocks 145432 00000020:00200000:2.0:1789760790.727591:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.727607:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.728345:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74448896, count: 4096 00000080:00200000:2.0:1789760790.728370:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74448896, bytes: 4096 00000080:00200000:2.0:1789760790.728389:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.728403:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.728406:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74448896, 74452992) 0 0 00000080:00200000:2.0:1789760790.728412:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.728441:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18176, 18176] 00020000:00200000:2.0:1789760790.728453:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74448896 chunk: [74448896, 74452992) 74452992, 4096 00020000:00200000:2.0:1789760790.728457:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.728483:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18176, 18176] 00000008:00010000:2.0:1789760790.728502:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.728509:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.728522:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74448896, 74452992) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.728536:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74448896=0x4700000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.728614:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.728617:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74452992 00000020:00010000:2.0:1789760790.728627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.728631:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.728634:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e2d8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.728638:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.728641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.728866:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.728873:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.728905:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760790.729048:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74448896<>74452992 00000100:00100000:3.0:1789760790.729162:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565557376:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.741611:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985484 gen 3 00000008:00010000:2.0:1789760790.741748:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760790.741757:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.741769:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.741866:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760790.741898:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565557376:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.742111:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18176, 18176] 00000080:00200000:1.0:1789760790.742128:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.742134:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.742150:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74452992 i_blocks 145440 00000020:00200000:1.0:1789760790.742154:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.742164:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.742198:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74452992, count: 4096 00000080:00200000:1.0:1789760790.742205:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74452992, bytes: 4096 00000080:00200000:1.0:1789760790.742216:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.742226:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.742228:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74452992, 74457088) 0 0 00000080:00200000:1.0:1789760790.742233:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.742252:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18177, 18177] 00020000:00200000:1.0:1789760790.742261:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74452992 chunk: [74452992, 74457088) 74457088, 4096 00020000:00200000:1.0:1789760790.742266:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.742285:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18177, 18177] 00000008:00010000:1.0:1789760790.742304:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.742310:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.742322:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74452992, 74457088) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.742336:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74452992=0x4701000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.742402:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.742406:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74457088 00000020:00010000:1.0:1789760790.742414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.742463:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.742466:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e2d8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.742470:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.742473:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.742482:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.742490:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.742509:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760790.742638:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74452992<>74457088 00000100:00100000:0.0:1789760790.742776:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565557504:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.753169:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985485 gen 3 00000008:00010000:1.0:1789760790.753277:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760790.753287:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.753299:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.753388:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760790.753428:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565557504:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.753826:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18177, 18177] 00000080:00200000:2.0:1789760790.753846:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.753852:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.753868:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74457088 i_blocks 145448 00000020:00200000:2.0:1789760790.753872:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.753883:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.753916:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74457088, count: 4096 00000080:00200000:2.0:1789760790.753924:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74457088, bytes: 4096 00000080:00200000:2.0:1789760790.753935:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.753944:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.753947:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74457088, 74461184) 0 0 00000080:00200000:2.0:1789760790.753950:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.753968:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18178, 18178] 00020000:00200000:2.0:1789760790.753977:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74457088 chunk: [74457088, 74461184) 74461184, 4096 00020000:00200000:2.0:1789760790.753981:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.754001:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18178, 18178] 00000008:00010000:2.0:1789760790.754025:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.754030:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.754043:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74457088, 74461184) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.754057:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74457088=0x4702000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.754123:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.754126:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74461184 00000020:00010000:2.0:1789760790.754135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.754139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.754142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e2d8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.754146:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0a88 (ffff93f84418a7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.754149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.754157:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.754164:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.754184:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760790.754915:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74457088<>74461184 00000100:00100000:2.0:1789760790.754973:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760790.754996:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565557632:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.769369:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985486 gen 3 00000008:00010000:2.0:1789760790.769436:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760790.769445:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.769457:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.769481:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760790.769516:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565557632:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760790.769822:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18178, 18178] 00000080:00200000:3.0:1789760790.769870:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760790.769876:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760790.769892:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74461184 i_blocks 145456 00000020:00200000:3.0:1789760790.769895:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760790.769904:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760790.769954:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74461184, count: 4096 00000080:00200000:3.0:1789760790.769963:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74461184, bytes: 4096 00000080:00200000:3.0:1789760790.769976:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760790.769984:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760790.769987:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74461184, 74465280) 0 0 00000080:00200000:3.0:1789760790.769991:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760790.770007:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18179, 18179] 00020000:00200000:3.0:1789760790.770016:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74461184 chunk: [74461184, 74465280) 74465280, 4096 00020000:00200000:3.0:1789760790.770021:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760790.770047:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18179, 18179] 00000008:00010000:3.0:1789760790.770066:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760790.770071:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760790.770083:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74461184, 74465280) append 0 append lockpos 0 00000080:00200000:3.0:1789760790.770096:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74461184=0x4703000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760790.770157:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760790.770160:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74465280 00000020:00010000:3.0:1789760790.770168:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760790.770171:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760790.770174:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760790.770178:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760790.770180:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760790.770191:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760790.770198:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760790.770214:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760790.770328:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 74461184<>74465280 00000100:00100000:3.0:1789760790.770373:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760790.770390:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565557760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.790092:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985487 gen 3 00000008:00010000:3.0:1789760790.790149:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:3.0:1789760790.790158:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.790169:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.790198:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:3.0:1789760790.790230:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565557760:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.790522:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18179, 18179] 00000080:00200000:2.0:1789760790.790540:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.790546:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.790562:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74465280 i_blocks 145464 00000020:00200000:2.0:1789760790.790566:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.790576:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.790612:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74465280, count: 4096 00000080:00200000:2.0:1789760790.790620:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74465280, bytes: 4096 00000080:00200000:2.0:1789760790.790632:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.790641:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.790690:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74465280, 74469376) 0 0 00000080:00200000:2.0:1789760790.790695:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.790714:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18180, 18180] 00020000:00200000:2.0:1789760790.790725:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74465280 chunk: [74465280, 74469376) 74469376, 4096 00020000:00200000:2.0:1789760790.790731:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.790752:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18180, 18180] 00000008:00010000:2.0:1789760790.790781:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.790787:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.790799:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74465280, 74469376) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.790814:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74465280=0x4704000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.790946:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.790949:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74469376 00000020:00010000:2.0:1789760790.790958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.790962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.790965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.790969:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.790972:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.790980:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.790986:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.791004:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760790.791133:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74465280<>74469376 00000100:00100000:2.0:1789760790.791256:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760790.791276:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565557888:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.803390:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985488 gen 3 00000008:00010000:2.0:1789760790.803450:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760790.803458:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.803470:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.803501:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760790.803530:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565557888:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.803739:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18180, 18180] 00000080:00200000:1.0:1789760790.803756:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.803830:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.803904:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74469376 i_blocks 145472 00000020:00200000:1.0:1789760790.803909:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.803920:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.803956:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74469376, count: 4096 00000080:00200000:1.0:1789760790.803965:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74469376, bytes: 4096 00000080:00200000:1.0:1789760790.803977:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.803986:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.803989:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74469376, 74473472) 0 0 00000080:00200000:1.0:1789760790.803993:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.804011:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18181, 18181] 00020000:00200000:1.0:1789760790.804020:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74469376 chunk: [74469376, 74473472) 74473472, 4096 00020000:00200000:1.0:1789760790.804024:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.804043:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18181, 18181] 00000008:00010000:1.0:1789760790.804063:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.804069:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.804081:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74469376, 74473472) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.804095:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74469376=0x4705000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.804447:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.804452:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74473472 00000020:00010000:1.0:1789760790.804461:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.804465:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.804469:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.804472:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.804475:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.804483:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.804550:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.804577:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760790.804753:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74469376<>74473472 00000100:00100000:1.0:1789760790.805260:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565558016:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.821062:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985489 gen 3 00000008:00010000:1.0:1789760790.821117:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760790.821126:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.821138:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.823008:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760790.823058:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565558016:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.823384:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18181, 18181] 00000080:00200000:2.0:1789760790.823402:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.823408:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.823433:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74473472 i_blocks 145480 00000020:00200000:2.0:1789760790.823439:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.823451:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.823488:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74473472, count: 4096 00000080:00200000:2.0:1789760790.823495:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74473472, bytes: 4096 00000080:00200000:2.0:1789760790.823510:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.823520:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.823522:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74473472, 74477568) 0 0 00000080:00200000:2.0:1789760790.823527:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.823545:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18182, 18182] 00020000:00200000:2.0:1789760790.823554:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74473472 chunk: [74473472, 74477568) 74477568, 4096 00020000:00200000:2.0:1789760790.823559:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.823580:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18182, 18182] 00000008:00010000:2.0:1789760790.823598:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.823605:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.823617:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74473472, 74477568) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.823630:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74473472=0x4706000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.823720:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.823724:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74477568 00000020:00010000:2.0:1789760790.823733:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.823736:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.823739:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.823742:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.823745:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.823754:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.823761:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.823779:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760790.823922:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 74473472<>74477568 00000100:00100000:3.0:1789760790.824450:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760790.824476:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565558144:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.838321:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985490 gen 3 00000008:00010000:3.0:1789760790.838383:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:3.0:1789760790.838392:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.838404:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.838433:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:3.0:1789760790.838465:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565558144:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.839924:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18182, 18182] 00000080:00200000:2.0:1789760790.839946:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.839953:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.839974:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74477568 i_blocks 145488 00000020:00200000:2.0:1789760790.839979:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.839993:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.840026:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74477568, count: 4096 00000080:00200000:2.0:1789760790.840034:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74477568, bytes: 4096 00000080:00200000:2.0:1789760790.840050:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.840059:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.840062:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74477568, 74481664) 0 0 00000080:00200000:2.0:1789760790.840066:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.840083:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18183, 18183] 00020000:00200000:2.0:1789760790.840092:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74477568 chunk: [74477568, 74481664) 74481664, 4096 00020000:00200000:2.0:1789760790.840098:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.840119:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18183, 18183] 00000008:00010000:2.0:1789760790.840137:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.840143:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.840271:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74477568, 74481664) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.840287:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74477568=0x4707000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.840365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.840369:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74481664 00000020:00010000:2.0:1789760790.840377:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.840381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.840384:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.840388:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.840391:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.840399:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.840406:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.840441:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760790.841272:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 74477568<>74481664 00000100:00100000:3.0:1789760790.841351:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565558272:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.854807:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985491 gen 3 00000008:00010000:3.0:1789760790.854963:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760790.854972:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.854983:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.855009:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760790.855043:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565558272:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.855289:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18183, 18183] 00000080:00200000:1.0:1789760790.855307:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.855314:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.855330:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74481664 i_blocks 145496 00000020:00200000:1.0:1789760790.855334:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.855345:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.855389:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74481664, count: 4096 00000080:00200000:1.0:1789760790.855397:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74481664, bytes: 4096 00000080:00200000:1.0:1789760790.855409:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.855426:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.855429:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74481664, 74485760) 0 0 00000080:00200000:1.0:1789760790.855434:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.855452:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18184, 18184] 00020000:00200000:1.0:1789760790.855461:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74481664 chunk: [74481664, 74485760) 74485760, 4096 00020000:00200000:1.0:1789760790.855467:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.855683:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18184, 18184] 00000008:00010000:1.0:1789760790.855705:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.855711:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.855722:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74481664, 74485760) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.855736:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74481664=0x4708000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.856329:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.856333:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74485760 00000020:00010000:1.0:1789760790.856342:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.856345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.856348:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.856360:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.856363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.856373:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.856380:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.856408:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760790.856709:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74481664<>74485760 00000100:00100000:0.0:1789760790.856792:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565558400:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.868006:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985492 gen 3 00000008:00010000:1.0:1789760790.868059:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760790.868069:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.868080:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.868103:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760790.868134:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565558400:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760790.868679:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18184, 18184] 00000080:00200000:3.0:1789760790.868701:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760790.868707:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760790.868728:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74485760 i_blocks 145504 00000020:00200000:3.0:1789760790.868733:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760790.868745:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760790.868794:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74485760, count: 4096 00000080:00200000:3.0:1789760790.868802:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74485760, bytes: 4096 00000080:00200000:3.0:1789760790.868814:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760790.868824:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760790.868827:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74485760, 74489856) 0 0 00000080:00200000:3.0:1789760790.868831:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760790.868849:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18185, 18185] 00020000:00200000:3.0:1789760790.868858:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74485760 chunk: [74485760, 74489856) 74489856, 4096 00020000:00200000:3.0:1789760790.868863:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760790.868886:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18185, 18185] 00000008:00010000:3.0:1789760790.868905:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760790.868913:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760790.868927:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74485760, 74489856) append 0 append lockpos 0 00000080:00200000:3.0:1789760790.868941:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74485760=0x4709000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760790.869004:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760790.869008:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74489856 00000020:00010000:3.0:1789760790.869016:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760790.869019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760790.869023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760790.869026:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760790.869030:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760790.869038:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760790.869046:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760790.869065:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760790.869847:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 74485760<>74489856 00000100:00100000:3.0:1789760790.869914:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760790.869938:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565558528:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760790.884793:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985493 gen 3 00000008:00010000:3.0:1789760790.884843:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:3.0:1789760790.884851:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760790.884863:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760790.884892:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:3.0:1789760790.884923:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565558528:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760790.885271:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18185, 18185] 00000080:00200000:1.0:1789760790.885291:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760790.885297:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760790.885313:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74489856 i_blocks 145512 00000020:00200000:1.0:1789760790.885316:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760790.885326:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760790.885374:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74489856, count: 4096 00000080:00200000:1.0:1789760790.885382:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74489856, bytes: 4096 00000080:00200000:1.0:1789760790.885393:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760790.885401:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760790.885403:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74489856, 74493952) 0 0 00000080:00200000:1.0:1789760790.885408:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760790.885435:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18186, 18186] 00020000:00200000:1.0:1789760790.885444:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74489856 chunk: [74489856, 74493952) 74493952, 4096 00020000:00200000:1.0:1789760790.885448:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760790.885467:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18186, 18186] 00000008:00010000:1.0:1789760790.885486:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760790.885493:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760790.885504:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74489856, 74493952) append 0 append lockpos 0 00000080:00200000:1.0:1789760790.885518:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74489856=0x470a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760790.885580:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760790.885584:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74493952 00000020:00010000:1.0:1789760790.885592:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:1.0:1789760790.885596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.885599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760790.885602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760790.885605:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760790.885613:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760790.885619:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.885682:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760790.885968:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74489856<>74493952 00000100:00100000:1.0:1789760790.886021:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565558656:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.898298:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985494 gen 3 00000008:00010000:1.0:1789760790.898349:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760790.898366:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.898378:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.899528:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760790.899577:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565558656:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760790.899831:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18186, 18186] 00000080:00200000:3.0:1789760790.899849:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760790.899855:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760790.899873:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74493952 i_blocks 145520 00000020:00200000:3.0:1789760790.899878:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760790.899889:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760790.899924:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74493952, count: 4096 00000080:00200000:3.0:1789760790.899933:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74493952, bytes: 4096 00000080:00200000:3.0:1789760790.899947:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760790.899956:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760790.899958:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74493952, 74498048) 0 0 00000080:00200000:3.0:1789760790.899963:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760790.899980:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18187, 18187] 00020000:00200000:3.0:1789760790.899988:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74493952 chunk: [74493952, 74498048) 74498048, 4096 00020000:00200000:3.0:1789760790.899992:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760790.900013:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18187, 18187] 00000008:00010000:3.0:1789760790.900033:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760790.900039:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760790.900052:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74493952, 74498048) append 0 append lockpos 0 00000080:00200000:3.0:1789760790.900065:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74493952=0x470b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760790.900130:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760790.900133:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74498048 00000020:00010000:3.0:1789760790.900142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:3.0:1789760790.900145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760790.900149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760790.900153:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760790.900156:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760790.900165:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760790.900172:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760790.900189:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760790.900480:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 74493952<>74498048 00000100:00100000:2.0:1789760790.900849:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565558784:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760790.910877:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985495 gen 3 00000008:00010000:2.0:1789760790.910929:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760790.910938:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760790.910950:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760790.910975:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760790.911006:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565558784:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.911124:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18187, 18187] 00000080:00200000:0.0:1789760790.911142:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.911148:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.911163:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74498048 i_blocks 145528 00000020:00200000:0.0:1789760790.911167:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.911177:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.911212:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74498048, count: 4096 00000080:00200000:0.0:1789760790.911219:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74498048, bytes: 4096 00000080:00200000:0.0:1789760790.911243:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.911252:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.911255:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74498048, 74502144) 0 0 00000080:00200000:0.0:1789760790.911259:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.911276:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18188, 18188] 00020000:00200000:0.0:1789760790.911285:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74498048 chunk: [74498048, 74502144) 74502144, 4096 00020000:00200000:0.0:1789760790.911290:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.911309:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18188, 18188] 00000008:00010000:0.0:1789760790.911325:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.911331:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.911342:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74498048, 74502144) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.911356:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74498048=0x470c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.911432:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.911435:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74502144 00000020:00010000:0.0:1789760790.911443:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.911446:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.911449:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.911453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.911456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.911464:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.911471:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.911488:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:0.0:1789760790.911614:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 74498048<>74502144 00000100:00100000:1.0:1789760790.912404:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760790.912445:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565558912:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.927567:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985496 gen 3 00000008:00010000:1.0:1789760790.927618:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760790.927627:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.927639:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.927694:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760790.927725:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565558912:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.928038:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18188, 18188] 00000080:00200000:0.0:1789760790.928056:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.928062:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.928076:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74502144 i_blocks 145536 00000020:00200000:0.0:1789760790.928081:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.928092:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.928125:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74502144, count: 4096 00000080:00200000:0.0:1789760790.928132:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74502144, bytes: 4096 00000080:00200000:0.0:1789760790.928141:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.928149:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.928151:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74502144, 74506240) 0 0 00000080:00200000:0.0:1789760790.928155:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.928170:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18189, 18189] 00020000:00200000:0.0:1789760790.928179:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74502144 chunk: [74502144, 74506240) 74506240, 4096 00020000:00200000:0.0:1789760790.928184:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.928202:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18189, 18189] 00000008:00010000:0.0:1789760790.928217:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.928223:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.928465:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74502144, 74506240) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.928481:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74502144=0x470d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.928555:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.928559:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74506240 00000020:00010000:0.0:1789760790.928567:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.928571:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.928573:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.928577:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.928580:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.928587:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.928595:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.928615:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760790.928783:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 74502144<>74506240 00000100:00100000:1.0:1789760790.929606:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565559040:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.949865:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985497 gen 3 00000008:00010000:1.0:1789760790.949918:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760790.949926:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.949937:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.950004:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:1.0:1789760790.950033:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565559040:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.950127:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18189, 18189] 00000080:00200000:0.0:1789760790.950144:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.950151:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.950167:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74506240 i_blocks 145544 00000020:00200000:0.0:1789760790.950171:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.950183:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.950217:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74506240, count: 4096 00000080:00200000:0.0:1789760790.950225:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74506240, bytes: 4096 00000080:00200000:0.0:1789760790.950287:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.950296:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.950299:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74506240, 74510336) 0 0 00000080:00200000:0.0:1789760790.950304:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.950323:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18190, 18190] 00020000:00200000:0.0:1789760790.950332:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74506240 chunk: [74506240, 74510336) 74510336, 4096 00020000:00200000:0.0:1789760790.950337:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.950358:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18190, 18190] 00000008:00010000:0.0:1789760790.950376:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.950382:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.950394:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74506240, 74510336) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.950408:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74506240=0x470e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.950886:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.950891:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74510336 00000020:00010000:0.0:1789760790.950901:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.950905:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.950907:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.950911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.950915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.950924:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.950932:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.950957:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760790.951086:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 74506240<>74510336 00000100:00100000:1.0:1789760790.951159:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565559168:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.960204:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985498 gen 3 00000008:00010000:1.0:1789760790.960258:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760790.960267:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.960280:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.960302:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760790.960333:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565559168:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.960450:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18190, 18190] 00000080:00200000:0.0:1789760790.960468:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.960473:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.960500:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74510336 i_blocks 145552 00000020:00200000:0.0:1789760790.960505:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.960516:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.960551:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74510336, count: 4096 00000080:00200000:0.0:1789760790.960559:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74510336, bytes: 4096 00000080:00200000:0.0:1789760790.960569:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.960577:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.960580:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74510336, 74514432) 0 0 00000080:00200000:0.0:1789760790.960585:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.960603:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18191, 18191] 00020000:00200000:0.0:1789760790.960612:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74510336 chunk: [74510336, 74514432) 74514432, 4096 00020000:00200000:0.0:1789760790.960617:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.960639:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18191, 18191] 00000008:00010000:0.0:1789760790.960705:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.960712:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.960723:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74510336, 74514432) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.960739:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74510336=0x470f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.961025:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.961028:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74514432 00000020:00010000:0.0:1789760790.961038:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.961042:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.961044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.961048:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.961051:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.961059:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.961066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760790.961218:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:1.0:1789760790.961348:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74510336<>74514432 00000100:00100000:1.0:1789760790.961559:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760790.961576:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565559296:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.982775:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985499 gen 3 00000008:00010000:1.0:1789760790.982824:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760790.982833:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.982843:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.982864:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:1.0:1789760790.982893:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565559296:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760790.982998:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18191, 18191] 00000080:00200000:0.0:1789760790.983014:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760790.983021:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760790.983037:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74514432 i_blocks 145560 00000020:00200000:0.0:1789760790.983041:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760790.983053:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760790.983089:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74514432, count: 4096 00000080:00200000:0.0:1789760790.983097:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74514432, bytes: 4096 00000080:00200000:0.0:1789760790.983108:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760790.983116:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760790.983119:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74514432, 74518528) 0 0 00000080:00200000:0.0:1789760790.983124:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760790.983141:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18192, 18192] 00020000:00200000:0.0:1789760790.983150:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74514432 chunk: [74514432, 74518528) 74518528, 4096 00020000:00200000:0.0:1789760790.983155:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760790.983175:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18192, 18192] 00000008:00010000:0.0:1789760790.983194:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760790.983200:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760790.983212:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74514432, 74518528) append 0 append lockpos 0 00000080:00200000:0.0:1789760790.983226:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74514432=0x4710000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760790.983284:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760790.983287:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74518528 00000020:00010000:0.0:1789760790.983296:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:0.0:1789760790.983299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.983302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760790.983306:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760790.983309:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760790.983316:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760790.983322:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760790.983338:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760790.984066:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41f80 - object 0x240000400:5 offset 74514432<>74518528 00000100:00100000:0.0:1789760790.984120:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760790.984141:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565559424:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760790.996752:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985500 gen 3 00000008:00010000:1.0:1789760790.996803:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760790.996811:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760790.996822:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760790.996846:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760790.996875:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565559424:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760790.997070:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18192, 18192] 00000080:00200000:2.0:1789760790.997088:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760790.997095:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760790.997111:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74518528 i_blocks 145568 00000020:00200000:2.0:1789760790.997115:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760790.997128:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760790.997163:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74518528, count: 4096 00000080:00200000:2.0:1789760790.997171:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74518528, bytes: 4096 00000080:00200000:2.0:1789760790.997181:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760790.997190:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760790.997192:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74518528, 74522624) 0 0 00000080:00200000:2.0:1789760790.997196:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760790.997213:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18193, 18193] 00020000:00200000:2.0:1789760790.997222:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74518528 chunk: [74518528, 74522624) 74522624, 4096 00020000:00200000:2.0:1789760790.997227:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760790.997248:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18193, 18193] 00000008:00010000:2.0:1789760790.997267:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760790.997273:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760790.997284:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74518528, 74522624) append 0 append lockpos 0 00000080:00200000:2.0:1789760790.997299:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74518528=0x4711000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760790.997358:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760790.997362:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74522624 00000020:00010000:2.0:1789760790.997370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:2.0:1789760790.997374:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.997377:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760790.997381:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760790.997384:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760790.997392:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760790.997400:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760790.997429:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760790.997623:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 74518528<>74522624 00000100:00100000:3.0:1789760790.997860:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565559552:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.008526:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985501 gen 3 00000008:00010000:2.0:1789760791.008751:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760791.008757:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.008765:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.008938:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760791.008961:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565559552:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.009180:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18193, 18193] 00000080:00200000:3.0:1789760791.009192:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.009196:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.009265:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74522624 i_blocks 145576 00000020:00200000:2.0:1789760791.009269:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.009277:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.009301:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74522624, count: 4096 00000080:00200000:2.0:1789760791.009307:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74522624, bytes: 4096 00000080:00200000:2.0:1789760791.009315:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.009323:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.009325:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74522624, 74526720) 0 0 00000080:00200000:2.0:1789760791.009329:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.009342:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18194, 18194] 00020000:00200000:2.0:1789760791.009348:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74522624 chunk: [74522624, 74526720) 74526720, 4096 00020000:00200000:2.0:1789760791.009352:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.009364:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18194, 18194] 00000008:00010000:2.0:1789760791.012311:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.012322:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.012335:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74522624, 74526720) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.012352:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74522624=0x4712000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.012409:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.012413:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74526720 00000020:00010000:2.0:1789760791.012431:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.012434:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.012437:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354ea28/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.012440:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.012442:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.012452:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.012459:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.012510:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760791.013172:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 74522624<>74526720 00000100:00100000:3.0:1789760791.013230:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760791.016371:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565559680:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.035580:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985502 gen 3 00000008:00010000:3.0:1789760791.035636:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760791.035689:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760791.035701:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760791.035790:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760791.035825:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565559680:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760791.036214:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18194, 18194] 00000080:00200000:1.0:1789760791.036233:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.036240:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.036256:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74526720 i_blocks 145584 00000020:00200000:1.0:1789760791.036260:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.036270:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.036306:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74526720, count: 4096 00000080:00200000:1.0:1789760791.036314:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74526720, bytes: 4096 00000080:00200000:1.0:1789760791.036325:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.036334:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.036337:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74526720, 74530816) 0 0 00000080:00200000:1.0:1789760791.036342:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.036727:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18195, 18195] 00020000:00200000:1.0:1789760791.036739:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74526720 chunk: [74526720, 74530816) 74530816, 4096 00020000:00200000:1.0:1789760791.036745:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.036839:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18195, 18195] 00000008:00010000:1.0:1789760791.036860:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.036867:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.036879:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74526720, 74530816) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.036893:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74526720=0x4713000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.036960:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.036963:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74530816 00000020:00010000:1.0:1789760791.036972:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.036975:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.036979:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354ea28/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.036982:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d04e8 (ffff93f84354e908/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.036985:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e908/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.036993:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.036999:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.037027:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760791.037157:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74526720<>74530816 00000100:00100000:0.0:1789760791.037242:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760791.037325:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565559808:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.052817:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985503 gen 3 00000008:00010000:1.0:1789760791.052941:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760791.052950:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.052962:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.053052:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760791.053328:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565559808:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.054925:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18195, 18195] 00000080:00200000:3.0:1789760791.054951:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.054958:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.054976:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74530816 i_blocks 145592 00000020:00200000:3.0:1789760791.054981:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.054994:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.055060:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74530816, count: 4096 00000080:00200000:3.0:1789760791.055070:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74530816, bytes: 4096 00000080:00200000:3.0:1789760791.055085:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.055095:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.055098:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74530816, 74534912) 0 0 00000080:00200000:3.0:1789760791.055102:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760791.055118:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18196, 18196] 00020000:00200000:3.0:1789760791.055128:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74530816 chunk: [74530816, 74534912) 74534912, 4096 00020000:00200000:3.0:1789760791.055132:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760791.055153:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18196, 18196] 00000008:00010000:3.0:1789760791.055405:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760791.055412:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760791.055437:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74530816, 74534912) append 0 append lockpos 0 00000080:00200000:3.0:1789760791.055452:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74530816=0x4714000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760791.055522:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760791.055525:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74534912 00000020:00010000:3.0:1789760791.055534:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:3.0:1789760791.055537:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.055540:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.055543:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760791.055546:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760791.055555:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760791.055561:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760791.055585:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760791.055794:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 74530816<>74534912 00000100:00100000:2.0:1789760791.056005:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760791.056031:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565559936:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.065785:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985504 gen 3 00000008:00010000:2.0:1789760791.065833:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760791.065842:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.065854:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.065877:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760791.065904:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565559936:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.066005:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18196, 18196] 00000080:00200000:3.0:1789760791.066023:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.066029:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.066045:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74534912 i_blocks 145600 00000020:00200000:3.0:1789760791.066049:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.066059:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.066092:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74534912, count: 4096 00000080:00200000:3.0:1789760791.066099:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74534912, bytes: 4096 00000080:00200000:3.0:1789760791.066107:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.066116:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.066118:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74534912, 74539008) 0 0 00000080:00200000:3.0:1789760791.066123:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760791.066139:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18197, 18197] 00020000:00200000:3.0:1789760791.066148:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74534912 chunk: [74534912, 74539008) 74539008, 4096 00020000:00200000:3.0:1789760791.066153:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760791.066172:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18197, 18197] 00000008:00010000:3.0:1789760791.066189:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760791.066195:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760791.066206:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74534912, 74539008) append 0 append lockpos 0 00000080:00200000:3.0:1789760791.066219:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74534912=0x4715000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760791.066274:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760791.066276:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74539008 00000020:00010000:3.0:1789760791.066283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:3.0:1789760791.066287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.066290:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.066294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760791.066297:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760791.066304:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760791.066310:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760791.066326:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760791.066453:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 74534912<>74539008 00000100:00100000:2.0:1789760791.066762:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565560064:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760791.067123:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760791.078725:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985505 gen 3 00000008:00010000:2.0:1789760791.078781:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760791.078790:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.078802:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.078821:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760791.078850:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565560064:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.078999:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18197, 18197] 00000080:00200000:3.0:1789760791.079015:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.079021:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.079036:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74539008 i_blocks 145608 00000020:00200000:3.0:1789760791.079039:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.079048:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.079081:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74539008, count: 4096 00000080:00200000:3.0:1789760791.079087:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74539008, bytes: 4096 00000080:00200000:3.0:1789760791.079097:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.079106:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.079108:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74539008, 74543104) 0 0 00000080:00200000:3.0:1789760791.079112:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760791.079128:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18198, 18198] 00020000:00200000:3.0:1789760791.079137:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74539008 chunk: [74539008, 74543104) 74543104, 4096 00020000:00200000:3.0:1789760791.079142:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760791.079160:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18198, 18198] 00000008:00010000:3.0:1789760791.079176:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760791.079182:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760791.079193:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74539008, 74543104) append 0 append lockpos 0 00000080:00200000:3.0:1789760791.079205:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74539008=0x4716000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760791.079258:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760791.079261:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74543104 00000020:00010000:3.0:1789760791.079269:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:3.0:1789760791.079272:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.079275:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.079279:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760791.079282:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760791.079289:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760791.079296:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760791.079312:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760791.079566:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66d80 - object 0x240000400:5 offset 74539008<>74543104 00000100:00100000:2.0:1789760791.079717:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760791.079735:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565560192:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.090016:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985506 gen 3 00000008:00010000:2.0:1789760791.090073:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760791.090081:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.090093:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.090165:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:2.0:1789760791.090193:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565560192:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.094343:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18198, 18198] 00000080:00200000:3.0:1789760791.094370:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.094376:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.094687:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74543104 i_blocks 145616 00000020:00200000:2.0:1789760791.094747:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.094765:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.094799:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74543104, count: 4096 00000080:00200000:2.0:1789760791.094808:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74543104, bytes: 4096 00000080:00200000:2.0:1789760791.094821:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.094830:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.094832:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74543104, 74547200) 0 0 00000080:00200000:2.0:1789760791.094837:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.094852:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18199, 18199] 00020000:00200000:2.0:1789760791.094861:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74543104 chunk: [74543104, 74547200) 74547200, 4096 00020000:00200000:2.0:1789760791.094866:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.094888:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18199, 18199] 00000008:00010000:2.0:1789760791.094904:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.094910:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.094921:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74543104, 74547200) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.094935:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74543104=0x4717000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.095005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.095008:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74547200 00000020:00010000:2.0:1789760791.095017:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.095021:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.095023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e3f8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.095027:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.095029:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.095038:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.095045:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.095100:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.095223:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 74543104<>74547200 00000100:00100000:3.0:1789760791.095314:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565560320:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.105029:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985507 gen 3 00000008:00010000:2.0:1789760791.105081:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.105091:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.105102:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.105131:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.105163:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565560320:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760791.105317:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18199, 18199] 00000080:00200000:1.0:1789760791.105334:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.105340:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.105366:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74547200 i_blocks 145624 00000020:00200000:1.0:1789760791.105371:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.105380:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.105523:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74547200, count: 4096 00000080:00200000:1.0:1789760791.105534:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74547200, bytes: 4096 00000080:00200000:1.0:1789760791.105545:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.105554:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.105557:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74547200, 74551296) 0 0 00000080:00200000:1.0:1789760791.105562:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.105578:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18200, 18200] 00020000:00200000:1.0:1789760791.105588:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74547200 chunk: [74547200, 74551296) 74551296, 4096 00020000:00200000:1.0:1789760791.105593:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.105614:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18200, 18200] 00000008:00010000:1.0:1789760791.105732:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.105739:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.105753:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74547200, 74551296) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.105768:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74547200=0x4718000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.105830:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.105833:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74551296 00000020:00010000:1.0:1789760791.105841:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.105845:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.105848:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.105852:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.105855:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.105862:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.105870:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.105889:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760791.106238:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74547200<>74551296 00000100:00100000:0.0:1789760791.106489:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760791.107273:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565560448:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.117141:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985508 gen 3 00000008:00010000:1.0:1789760791.117196:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760791.117204:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.117215:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.117297:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760791.117326:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565560448:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.119162:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18200, 18200] 00000080:00200000:3.0:1789760791.119189:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.119196:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.119329:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74551296 i_blocks 145632 00000020:00200000:1.0:1789760791.119572:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.119588:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.119623:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74551296, count: 4096 00000080:00200000:1.0:1789760791.119632:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74551296, bytes: 4096 00000080:00200000:1.0:1789760791.119665:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.119675:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.119678:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74551296, 74555392) 0 0 00000080:00200000:1.0:1789760791.119683:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.119701:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18201, 18201] 00020000:00200000:1.0:1789760791.119709:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74551296 chunk: [74551296, 74555392) 74555392, 4096 00020000:00200000:1.0:1789760791.119714:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.119737:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18201, 18201] 00000008:00010000:1.0:1789760791.119755:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.119899:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.119915:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74551296, 74555392) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.119930:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74551296=0x4719000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.120068:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.120071:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74555392 00000020:00010000:1.0:1789760791.120158:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.120162:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.120166:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.120170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.120173:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.120181:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.120188:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.120217:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760791.120489:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74551296<>74555392 00000100:00100000:0.0:1789760791.120581:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760791.120838:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565560576:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.132264:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985509 gen 3 00000008:00010000:1.0:1789760791.132314:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760791.132324:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.132334:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.132369:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760791.132399:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565560576:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.132612:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18201, 18201] 00000080:00200000:2.0:1789760791.132630:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.132636:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.132683:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74555392 i_blocks 145640 00000020:00200000:2.0:1789760791.132687:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.132698:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.132734:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74555392, count: 4096 00000080:00200000:2.0:1789760791.132743:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74555392, bytes: 4096 00000080:00200000:2.0:1789760791.132755:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.132774:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.132777:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74555392, 74559488) 0 0 00000080:00200000:2.0:1789760791.132781:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.132800:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18202, 18202] 00020000:00200000:2.0:1789760791.132809:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74555392 chunk: [74555392, 74559488) 74559488, 4096 00020000:00200000:2.0:1789760791.132813:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.132834:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18202, 18202] 00000008:00010000:2.0:1789760791.132852:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.132857:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.132868:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74555392, 74559488) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.132880:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74555392=0x471a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.132931:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.132934:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74559488 00000020:00010000:2.0:1789760791.132942:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.132946:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.132949:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.132952:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.132955:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.132962:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.132968:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.132984:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.133108:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 74555392<>74559488 00000100:00100000:2.0:1789760791.133158:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760791.133175:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565560704:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.145142:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985510 gen 3 00000008:00010000:2.0:1789760791.145197:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.145206:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.145218:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.145247:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.145280:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565560704:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760791.145730:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18202, 18202] 00000080:00200000:1.0:1789760791.145749:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.145755:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.145783:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74559488 i_blocks 145648 00000020:00200000:1.0:1789760791.145787:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.145799:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.145837:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74559488, count: 4096 00000080:00200000:1.0:1789760791.145845:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74559488, bytes: 4096 00000080:00200000:1.0:1789760791.145856:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.145865:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.145867:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74559488, 74563584) 0 0 00000080:00200000:1.0:1789760791.145872:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.145891:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18203, 18203] 00020000:00200000:1.0:1789760791.145900:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74559488 chunk: [74559488, 74563584) 74563584, 4096 00020000:00200000:1.0:1789760791.145905:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.145927:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18203, 18203] 00000008:00010000:1.0:1789760791.145946:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.145953:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.145966:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74559488, 74563584) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.145982:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74559488=0x471b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.146046:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.146049:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74563584 00000020:00010000:1.0:1789760791.146057:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.146060:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.146064:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.146067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.146071:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.146078:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.146085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.146104:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760791.146232:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74559488<>74563584 00000100:00100000:1.0:1789760791.146292:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565560832:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.156177:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985511 gen 3 00000008:00010000:1.0:1789760791.156229:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760791.156238:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.156248:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.156277:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760791.156308:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565560832:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.156333:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18203, 18203] 00000080:00200000:2.0:1789760791.157325:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.157336:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.157467:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74563584 i_blocks 145656 00000020:00200000:2.0:1789760791.157473:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.157486:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.157522:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74563584, count: 4096 00000080:00200000:2.0:1789760791.157530:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74563584, bytes: 4096 00000080:00200000:2.0:1789760791.157544:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.158197:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.158201:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74563584, 74567680) 0 0 00000080:00200000:2.0:1789760791.158206:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.158227:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18204, 18204] 00020000:00200000:2.0:1789760791.158237:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74563584 chunk: [74563584, 74567680) 74567680, 4096 00020000:00200000:2.0:1789760791.158243:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.158268:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18204, 18204] 00000008:00010000:2.0:1789760791.158289:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.158295:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.158308:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74563584, 74567680) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.158324:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74563584=0x471c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.158535:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.158539:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74567680 00000020:00010000:2.0:1789760791.158685:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.158755:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.158836:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.158840:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.158843:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.158853:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.158861:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.158891:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.159097:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 74563584<>74567680 00000100:00100000:3.0:1789760791.159196:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565560960:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.171906:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985512 gen 3 00000008:00010000:2.0:1789760791.173366:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.173379:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.173391:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.173429:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.173470:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565560960:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760791.173736:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18204, 18204] 00000080:00200000:1.0:1789760791.173753:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.173768:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.173785:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74567680 i_blocks 145664 00000020:00200000:1.0:1789760791.173789:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.173799:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.173837:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74567680, count: 4096 00000080:00200000:1.0:1789760791.173844:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74567680, bytes: 4096 00000080:00200000:1.0:1789760791.173859:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.173868:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.173870:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74567680, 74571776) 0 0 00000080:00200000:1.0:1789760791.173875:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.173893:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18205, 18205] 00020000:00200000:1.0:1789760791.173902:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74567680 chunk: [74567680, 74571776) 74571776, 4096 00020000:00200000:1.0:1789760791.173907:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.173926:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18205, 18205] 00000008:00010000:1.0:1789760791.173946:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.173952:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.173964:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74567680, 74571776) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.173986:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74567680=0x471d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.174050:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.174053:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74571776 00000020:00010000:1.0:1789760791.174061:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.174065:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.174068:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.174071:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.174074:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.174082:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.174088:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.174106:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760791.174229:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74567680<>74571776 00000100:00100000:1.0:1789760791.174282:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760791.174302:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565561088:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.188125:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985513 gen 3 00000008:00010000:1.0:1789760791.188175:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:1.0:1789760791.188184:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.188193:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.188228:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:1.0:1789760791.188259:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565561088:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.189732:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18205, 18205] 00000080:00200000:2.0:1789760791.189758:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.189765:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.189785:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74571776 i_blocks 145672 00000020:00200000:2.0:1789760791.189789:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.189804:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.189838:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74571776, count: 4096 00000080:00200000:2.0:1789760791.189847:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74571776, bytes: 4096 00000080:00200000:2.0:1789760791.189861:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.189871:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.189873:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74571776, 74575872) 0 0 00000080:00200000:2.0:1789760791.189879:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.189898:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18206, 18206] 00020000:00200000:2.0:1789760791.189907:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74571776 chunk: [74571776, 74575872) 74575872, 4096 00020000:00200000:2.0:1789760791.189924:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.189947:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18206, 18206] 00000008:00010000:2.0:1789760791.189975:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.189982:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.189994:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74571776, 74575872) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.190008:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74571776=0x471e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.190074:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.190078:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74575872 00000020:00010000:2.0:1789760791.190086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.190089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.190092:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.190096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.190099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.190107:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.190114:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.190134:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.190272:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 74571776<>74575872 00000100:00100000:2.0:1789760791.190327:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760791.190424:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565561216:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.204002:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985514 gen 3 00000008:00010000:2.0:1789760791.204055:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.204064:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.204076:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.204145:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.204176:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565561216:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.208612:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18206, 18206] 00000080:00200000:3.0:1789760791.208694:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.208701:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.210924:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74575872 i_blocks 145680 00000020:00200000:2.0:1789760791.210939:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.210957:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.210994:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74575872, count: 4096 00000080:00200000:2.0:1789760791.211005:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74575872, bytes: 4096 00000080:00200000:2.0:1789760791.211020:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.211029:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.211031:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74575872, 74579968) 0 0 00000080:00200000:2.0:1789760791.211037:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.211055:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18207, 18207] 00020000:00200000:2.0:1789760791.211066:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74575872 chunk: [74575872, 74579968) 74579968, 4096 00020000:00200000:2.0:1789760791.211071:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.211095:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18207, 18207] 00000008:00010000:2.0:1789760791.211113:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.211119:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.211132:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74575872, 74579968) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.211146:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74575872=0x471f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.211282:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.211285:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74579968 00000020:00010000:2.0:1789760791.211294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.211299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.211302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e6c8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.211305:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e998 (ffff93f84354ea28/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.211309:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354ea28/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.211318:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.211325:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.211384:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.212147:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 74575872<>74579968 00000100:00100000:2.0:1789760791.212229:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565561344:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.225197:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985515 gen 3 00000008:00010000:2.0:1789760791.225249:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.225258:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.225270:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.225293:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.225325:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565561344:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.225457:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18207, 18207] 00000080:00200000:3.0:1789760791.225502:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.225508:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.225524:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74579968 i_blocks 145688 00000020:00200000:3.0:1789760791.225528:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.225537:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.225586:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74579968, count: 4096 00000080:00200000:3.0:1789760791.225595:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74579968, bytes: 4096 00000080:00200000:3.0:1789760791.225616:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.225625:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.225628:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74579968, 74584064) 0 0 00000080:00200000:3.0:1789760791.225632:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.228518:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18208, 18208] 00020000:00200000:0.0:1789760791.228538:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74579968 chunk: [74579968, 74584064) 74584064, 4096 00020000:00200000:0.0:1789760791.228543:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.228573:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18208, 18208] 00000008:00010000:0.0:1789760791.228600:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.228607:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.228621:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74579968, 74584064) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.228672:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74579968=0x4720000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.228751:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.228755:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74584064 00000020:00010000:0.0:1789760791.228764:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.228768:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.228770:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.228773:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.228776:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.228787:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.228800:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.228818:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760791.228961:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43100 - object 0x240000400:5 offset 74579968<>74584064 00000100:00100000:0.0:1789760791.229017:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760791.229042:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565561472:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.246112:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985516 gen 3 00000008:00010000:1.0:1789760791.246156:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760791.246164:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.246174:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.246193:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760791.246220:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565561472:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.246314:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18208, 18208] 00000080:00200000:0.0:1789760791.246329:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.246335:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.246348:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74584064 i_blocks 145696 00000020:00200000:0.0:1789760791.246351:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.246360:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.246393:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74584064, count: 4096 00000080:00200000:0.0:1789760791.246401:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74584064, bytes: 4096 00000080:00200000:0.0:1789760791.246409:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.246437:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.246440:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74584064, 74588160) 0 0 00000080:00200000:0.0:1789760791.246445:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.246462:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18209, 18209] 00020000:00200000:0.0:1789760791.246470:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74584064 chunk: [74584064, 74588160) 74588160, 4096 00020000:00200000:0.0:1789760791.246475:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.246495:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18209, 18209] 00000008:00010000:0.0:1789760791.246511:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.246516:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.246527:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74584064, 74588160) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.246540:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74584064=0x4721000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.246598:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.246601:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74588160 00000020:00010000:0.0:1789760791.246609:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.246613:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.246616:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.246620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.246623:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.246631:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.246637:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.246685:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760791.246812:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40700 - object 0x240000400:5 offset 74584064<>74588160 00000100:00100000:1.0:1789760791.247123:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565561600:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.265804:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985517 gen 3 00000008:00010000:1.0:1789760791.265856:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760791.265864:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.265875:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.265901:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:1.0:1789760791.265934:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565561600:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.266059:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18209, 18209] 00000080:00200000:0.0:1789760791.266076:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.266083:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.266097:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74588160 i_blocks 145704 00000020:00200000:0.0:1789760791.266101:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.266115:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.266146:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74588160, count: 4096 00000080:00200000:0.0:1789760791.266153:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74588160, bytes: 4096 00000080:00200000:0.0:1789760791.266167:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.266176:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.266178:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74588160, 74592256) 0 0 00000080:00200000:0.0:1789760791.266182:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.266200:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18210, 18210] 00020000:00200000:0.0:1789760791.266208:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74588160 chunk: [74588160, 74592256) 74592256, 4096 00020000:00200000:0.0:1789760791.266213:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.266233:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18210, 18210] 00000008:00010000:0.0:1789760791.266251:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.266257:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.266268:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74588160, 74592256) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.266281:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74588160=0x4722000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.266340:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.266344:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74592256 00000020:00010000:0.0:1789760791.266353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.266356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.266358:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.266362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.266365:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.266371:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.266378:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.266395:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d300 00000008:00100000:0.0:1789760791.266537:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 74588160<>74592256 00000100:00100000:1.0:1789760791.266833:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760791.266852:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565561728:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.279684:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985518 gen 3 00000008:00010000:1.0:1789760791.279736:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760791.279745:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.279757:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.279779:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760791.279809:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565561728:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.279919:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18210, 18210] 00000080:00200000:0.0:1789760791.279936:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.279942:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.279957:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74592256 i_blocks 145712 00000020:00200000:0.0:1789760791.279961:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.279973:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.280008:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74592256, count: 4096 00000080:00200000:0.0:1789760791.280016:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74592256, bytes: 4096 00000080:00200000:0.0:1789760791.280026:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.280035:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.280038:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74592256, 74596352) 0 0 00000080:00200000:0.0:1789760791.280043:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.280062:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18211, 18211] 00020000:00200000:0.0:1789760791.280070:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74592256 chunk: [74592256, 74596352) 74596352, 4096 00020000:00200000:0.0:1789760791.280075:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.280095:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18211, 18211] 00000008:00010000:0.0:1789760791.280113:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.280119:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.280130:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74592256, 74596352) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.280143:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74592256=0x4723000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.280200:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.280204:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74596352 00000020:00010000:0.0:1789760791.280213:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.280216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.280220:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.280223:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.280226:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.280234:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.280240:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.280257:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760791.280391:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42680 - object 0x240000400:5 offset 74592256<>74596352 00000100:00100000:1.0:1789760791.280613:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565561856:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.290007:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985519 gen 3 00000008:00010000:1.0:1789760791.290060:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760791.290069:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.290081:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.290098:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000080:00200000:0.0:1789760791.290132:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18211, 18211] 00000100:00100000:1.0:1789760791.290133:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565561856:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.290154:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.290436:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.290455:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74596352 i_blocks 145720 00000020:00200000:0.0:1789760791.290459:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.290471:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.290560:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74596352, count: 4096 00000080:00200000:0.0:1789760791.290569:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74596352, bytes: 4096 00000080:00200000:0.0:1789760791.290580:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.290588:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.290590:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74596352, 74600448) 0 0 00000080:00200000:0.0:1789760791.290596:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.290988:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18212, 18212] 00020000:00200000:0.0:1789760791.290998:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74596352 chunk: [74596352, 74600448) 74600448, 4096 00020000:00200000:0.0:1789760791.291003:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.291026:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18212, 18212] 00000008:00010000:0.0:1789760791.291044:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.291050:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.291063:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74596352, 74600448) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.291079:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74596352=0x4724000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.291146:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.291149:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74600448 00000020:00010000:0.0:1789760791.291158:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.291161:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.291164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.291168:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.291171:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.291179:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.291187:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.291211:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760791.291344:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 74596352<>74600448 00000100:00100000:1.0:1789760791.292483:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760791.292513:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565561984:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.309878:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985520 gen 3 00000008:00010000:1.0:1789760791.309929:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760791.309938:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.309949:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.309969:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760791.309999:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565561984:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.310105:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18212, 18212] 00000080:00200000:0.0:1789760791.310122:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.310128:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.310144:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74600448 i_blocks 145728 00000020:00200000:0.0:1789760791.310148:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.310159:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.310192:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74600448, count: 4096 00000080:00200000:0.0:1789760791.310200:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74600448, bytes: 4096 00000080:00200000:0.0:1789760791.310209:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.310218:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.310220:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74600448, 74604544) 0 0 00000080:00200000:0.0:1789760791.310225:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.310243:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18213, 18213] 00020000:00200000:0.0:1789760791.310252:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74600448 chunk: [74600448, 74604544) 74604544, 4096 00020000:00200000:0.0:1789760791.310256:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.310276:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18213, 18213] 00000008:00010000:0.0:1789760791.310293:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.310299:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.310310:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74600448, 74604544) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.310324:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74600448=0x4725000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.310381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.310385:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74604544 00000020:00010000:0.0:1789760791.310393:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.310397:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.310399:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.310403:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.310405:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.310422:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.310509:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.310527:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760791.310774:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 74600448<>74604544 00000100:00100000:1.0:1789760791.310829:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565562112:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.323737:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985521 gen 3 00000008:00010000:1.0:1789760791.323791:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760791.323801:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.323812:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.323840:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760791.323872:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565562112:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.323901:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18213, 18213] 00000080:00200000:2.0:1789760791.323919:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.323925:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.323942:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74604544 i_blocks 145736 00000020:00200000:2.0:1789760791.323946:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.323959:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.323995:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74604544, count: 4096 00000080:00200000:2.0:1789760791.324003:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74604544, bytes: 4096 00000080:00200000:2.0:1789760791.324024:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.324034:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.324036:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74604544, 74608640) 0 0 00000080:00200000:2.0:1789760791.324041:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.324060:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18214, 18214] 00020000:00200000:2.0:1789760791.324071:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74604544 chunk: [74604544, 74608640) 74608640, 4096 00020000:00200000:2.0:1789760791.324076:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.324096:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18214, 18214] 00000008:00010000:2.0:1789760791.324115:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.324122:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.324143:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74604544, 74608640) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.324157:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74604544=0x4726000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.324216:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.324219:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74608640 00000020:00010000:2.0:1789760791.324229:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.324232:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.324235:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.324239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.324243:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.324251:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.324258:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.324274:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.324395:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 74604544<>74608640 00000100:00100000:2.0:1789760791.324450:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760791.324466:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565562240:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.351786:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985522 gen 3 00000008:00010000:2.0:1789760791.351839:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.351848:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.351860:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.351888:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.351920:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565562240:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760791.352044:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18214, 18214] 00000080:00200000:1.0:1789760791.352062:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.352067:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.352083:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74608640 i_blocks 145744 00000020:00200000:1.0:1789760791.352087:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.352098:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.352134:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74608640, count: 4096 00000080:00200000:1.0:1789760791.352142:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74608640, bytes: 4096 00000080:00200000:1.0:1789760791.352153:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.352161:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.352163:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74608640, 74612736) 0 0 00000080:00200000:1.0:1789760791.352167:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.352184:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18215, 18215] 00020000:00200000:1.0:1789760791.352192:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74608640 chunk: [74608640, 74612736) 74612736, 4096 00020000:00200000:1.0:1789760791.352196:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.352215:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18215, 18215] 00000008:00010000:1.0:1789760791.352232:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.352238:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.352250:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74608640, 74612736) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.352263:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74608640=0x4727000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.352320:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.352323:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74612736 00000020:00010000:1.0:1789760791.352332:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.352335:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.352338:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.352341:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.352344:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.352362:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.352369:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.352388:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760791.352519:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 74608640<>74612736 00000100:00100000:1.0:1789760791.352560:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760791.352576:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565562368:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760791.374782:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985523 gen 3 00000008:00010000:0.0:1789760791.374835:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760791.374843:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760791.374854:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760791.374871:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000080:00200000:2.0:1789760791.374892:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18215, 18215] 00000100:00100000:0.0:1789760791.374895:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565562368:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.374909:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.374915:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.374930:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74612736 i_blocks 145752 00000020:00200000:2.0:1789760791.374934:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.374944:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.374977:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74612736, count: 4096 00000080:00200000:2.0:1789760791.374984:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74612736, bytes: 4096 00000080:00200000:2.0:1789760791.374994:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.375003:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.375006:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74612736, 74616832) 0 0 00000080:00200000:2.0:1789760791.375012:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.375029:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18216, 18216] 00020000:00200000:2.0:1789760791.375038:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74612736 chunk: [74612736, 74616832) 74616832, 4096 00020000:00200000:2.0:1789760791.375042:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.375060:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18216, 18216] 00000008:00010000:2.0:1789760791.375078:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.375085:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.375096:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74612736, 74616832) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.375109:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74612736=0x4728000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.375167:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.375170:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74616832 00000020:00010000:2.0:1789760791.375178:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.375182:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.375184:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.375188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.375191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.375198:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.375205:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.375219:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.375346:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 74612736<>74616832 00000100:00100000:3.0:1789760791.375611:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565562496:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.388500:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985524 gen 3 00000008:00010000:3.0:1789760791.388550:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:3.0:1789760791.388558:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760791.388568:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760791.388591:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:3.0:1789760791.388620:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565562496:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.389534:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18216, 18216] 00000080:00200000:2.0:1789760791.389554:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.389562:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.389579:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74616832 i_blocks 145760 00000020:00200000:2.0:1789760791.389583:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.389594:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.389631:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74616832, count: 4096 00000080:00200000:2.0:1789760791.389639:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74616832, bytes: 4096 00000080:00200000:2.0:1789760791.389677:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.389687:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.389689:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74616832, 74620928) 0 0 00000080:00200000:2.0:1789760791.389693:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.389709:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18217, 18217] 00020000:00200000:2.0:1789760791.389718:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74616832 chunk: [74616832, 74620928) 74620928, 4096 00020000:00200000:2.0:1789760791.389723:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.389745:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18217, 18217] 00000008:00010000:2.0:1789760791.389765:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.389771:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.389784:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74616832, 74620928) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.389800:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74616832=0x4729000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.389889:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.389892:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74620928 00000020:00010000:2.0:1789760791.389901:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.389904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.389908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.389911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.389914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.389923:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.389930:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.389949:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760791.390079:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 74616832<>74620928 00000100:00100000:3.0:1789760791.390348:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565562624:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.402353:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985525 gen 3 00000008:00010000:3.0:1789760791.402404:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760791.402422:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760791.402434:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760791.402460:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760791.402498:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565562624:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.402624:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18217, 18217] 00000080:00200000:2.0:1789760791.402674:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.402681:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.402695:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74620928 i_blocks 145768 00000020:00200000:2.0:1789760791.402699:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.402710:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.403070:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74620928, count: 4096 00000080:00200000:2.0:1789760791.403077:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74620928, bytes: 4096 00000080:00200000:2.0:1789760791.403087:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.403094:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.403096:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74620928, 74625024) 0 0 00000080:00200000:2.0:1789760791.403100:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.403114:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18218, 18218] 00020000:00200000:2.0:1789760791.403122:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74620928 chunk: [74620928, 74625024) 74625024, 4096 00020000:00200000:2.0:1789760791.403126:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.403143:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18218, 18218] 00000008:00010000:2.0:1789760791.403162:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.403179:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.403192:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74620928, 74625024) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.403205:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74620928=0x472a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.403263:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.403265:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74625024 00000020:00010000:2.0:1789760791.403273:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.403275:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.403278:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.403280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.403282:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.403287:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.403292:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.403309:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760791.405327:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 74620928<>74625024 00000100:00100000:3.0:1789760791.405425:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565562752:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.420726:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985526 gen 3 00000008:00010000:3.0:1789760791.420781:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760791.420790:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760791.420801:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760791.420836:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000080:00200000:1.0:1789760791.421029:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18218, 18218] 00000080:00200000:1.0:1789760791.421047:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.421054:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.421068:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74625024 i_blocks 145776 00000020:00200000:1.0:1789760791.421071:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.421080:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.421114:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74625024, count: 4096 00000080:00200000:1.0:1789760791.421123:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74625024, bytes: 4096 00000080:00200000:1.0:1789760791.421133:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.421142:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.421144:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74625024, 74629120) 0 0 00000080:00200000:1.0:1789760791.421149:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.421166:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18219, 18219] 00020000:00200000:1.0:1789760791.421175:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74625024 chunk: [74625024, 74629120) 74629120, 4096 00020000:00200000:1.0:1789760791.421181:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.421200:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18219, 18219] 00000008:00010000:1.0:1789760791.421217:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.421224:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.421234:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74625024, 74629120) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.421247:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74625024=0x472b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.421295:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.421298:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74629120 00000020:00010000:1.0:1789760791.421306:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.421310:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.421313:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.421316:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.421319:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.421325:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.421332:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:3.0:1789760791.421462:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565562752:192.168.203.147@tcp:4:dd.0 00000008:00010000:3.0:1789760791.421508:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:3.0:1789760791.421631:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 74625024<>74629120 00000100:00100000:3.0:1789760791.421699:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760791.421713:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565562880:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.436516:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985527 gen 3 00000008:00010000:3.0:1789760791.436569:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:3.0:1789760791.436578:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760791.436588:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760791.436694:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:3.0:1789760791.436727:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565562880:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760791.438130:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18219, 18219] 00000080:00200000:1.0:1789760791.438155:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.438162:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.438182:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74629120 i_blocks 145784 00000020:00200000:1.0:1789760791.438187:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.438202:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.438241:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74629120, count: 4096 00000080:00200000:1.0:1789760791.438250:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74629120, bytes: 4096 00000080:00200000:1.0:1789760791.438267:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.438277:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.438280:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74629120, 74633216) 0 0 00000080:00200000:1.0:1789760791.438284:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.438303:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18220, 18220] 00020000:00200000:1.0:1789760791.438314:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74629120 chunk: [74629120, 74633216) 74633216, 4096 00020000:00200000:1.0:1789760791.438318:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.438341:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18220, 18220] 00000008:00010000:1.0:1789760791.438361:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.438369:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.438381:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74629120, 74633216) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.438396:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74629120=0x472c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.438571:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.438575:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74633216 00000020:00010000:1.0:1789760791.438585:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.438589:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.438592:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e7e8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.438596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.438599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.438607:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.438614:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.438667:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760791.438802:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74629120<>74633216 00000100:00100000:0.0:1789760791.438911:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565563008:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.450978:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985528 gen 3 00000008:00010000:1.0:1789760791.451034:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760791.451043:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.451055:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.451132:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760791.451161:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565563008:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.452093:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18220, 18220] 00000080:00200000:3.0:1789760791.452116:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.452123:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.452294:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74633216 i_blocks 145792 00000020:00200000:2.0:1789760791.452300:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.452313:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.452345:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74633216, count: 4096 00000080:00200000:2.0:1789760791.452409:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74633216, bytes: 4096 00000080:00200000:2.0:1789760791.452431:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.452439:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.452442:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74633216, 74637312) 0 0 00000080:00200000:2.0:1789760791.452446:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.452461:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18221, 18221] 00020000:00200000:2.0:1789760791.452470:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74633216 chunk: [74633216, 74637312) 74637312, 4096 00020000:00200000:2.0:1789760791.452474:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.452551:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18221, 18221] 00000008:00010000:2.0:1789760791.452570:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.452576:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.452589:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74633216, 74637312) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.452603:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74633216=0x472d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.452691:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.452695:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74637312 00000020:00010000:2.0:1789760791.452704:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.452707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.452710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e518/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.452714:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.452717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.452725:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.452733:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.452860:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.452987:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 74633216<>74637312 00000100:00100000:3.0:1789760791.453064:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565563136:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.464122:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985529 gen 3 00000008:00010000:2.0:1789760791.464177:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.464186:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.464198:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.464228:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.464260:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565563136:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760791.464402:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18221, 18221] 00000080:00200000:1.0:1789760791.464428:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.464435:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.464451:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74637312 i_blocks 145800 00000020:00200000:1.0:1789760791.464456:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.464467:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.464502:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74637312, count: 4096 00000080:00200000:1.0:1789760791.464509:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74637312, bytes: 4096 00000080:00200000:1.0:1789760791.464521:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.464530:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.464533:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74637312, 74641408) 0 0 00000080:00200000:1.0:1789760791.464548:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.464566:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18222, 18222] 00020000:00200000:1.0:1789760791.464575:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74637312 chunk: [74637312, 74641408) 74641408, 4096 00020000:00200000:1.0:1789760791.464580:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.464600:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18222, 18222] 00000008:00010000:1.0:1789760791.464618:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.464623:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.464635:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74637312, 74641408) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.464678:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74637312=0x472e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.464751:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.464754:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74641408 00000020:00010000:1.0:1789760791.464762:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.464765:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.464768:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e518/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.464772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.464775:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.464781:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.464788:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.464805:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760791.464929:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74637312<>74641408 00000100:00100000:1.0:1789760791.464976:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760791.464993:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565563264:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.482467:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985530 gen 3 00000008:00010000:1.0:1789760791.482527:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760791.482537:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.482548:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.482576:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760791.482607:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565563264:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.482617:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18222, 18222] 00000080:00200000:2.0:1789760791.482635:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.482641:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.482911:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74641408 i_blocks 145808 00000020:00200000:2.0:1789760791.482916:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.482926:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.482962:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74641408, count: 4096 00000080:00200000:2.0:1789760791.482979:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74641408, bytes: 4096 00000080:00200000:2.0:1789760791.482992:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.483008:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.483010:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74641408, 74645504) 0 0 00000080:00200000:2.0:1789760791.483015:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.483034:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18223, 18223] 00020000:00200000:2.0:1789760791.483043:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74641408 chunk: [74641408, 74645504) 74645504, 4096 00020000:00200000:2.0:1789760791.483309:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.483329:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18223, 18223] 00000008:00010000:2.0:1789760791.483345:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.483350:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.483362:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74641408, 74645504) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.483377:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74641408=0x472f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.483452:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.483455:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74645504 00000020:00010000:2.0:1789760791.483463:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.483467:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.483470:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e518/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.483474:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ed58 (ffff93f84354e3f8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.483477:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e3f8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.483703:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.483712:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.483736:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.483863:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 74641408<>74645504 00000100:00100000:2.0:1789760791.483920:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760791.483941:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565563392:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.501347:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985531 gen 3 00000008:00010000:2.0:1789760791.501470:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.501479:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.501545:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.501616:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.501700:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565563392:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.501961:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18223, 18223] 00000080:00200000:3.0:1789760791.502008:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.502015:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.502031:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74645504 i_blocks 145816 00000020:00200000:3.0:1789760791.502035:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.502044:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.502098:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74645504, count: 4096 00000080:00200000:3.0:1789760791.502106:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74645504, bytes: 4096 00000080:00200000:3.0:1789760791.502118:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.502127:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.502130:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74645504, 74649600) 0 0 00000080:00200000:3.0:1789760791.502135:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760791.502153:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18224, 18224] 00020000:00200000:3.0:1789760791.502161:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74645504 chunk: [74645504, 74649600) 74649600, 4096 00020000:00200000:3.0:1789760791.502165:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760791.502185:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18224, 18224] 00000008:00010000:3.0:1789760791.502202:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760791.502209:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760791.502221:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74645504, 74649600) append 0 append lockpos 0 00000080:00200000:3.0:1789760791.502236:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74645504=0x4730000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760791.502300:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760791.502303:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74649600 00000020:00010000:3.0:1789760791.502313:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760791.502317:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.502320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.502323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760791.502326:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760791.502335:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760791.502343:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760791.502433:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760791.502613:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 74645504<>74649600 00000100:00100000:2.0:1789760791.502686:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565563520:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.512160:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985532 gen 3 00000008:00010000:2.0:1789760791.512210:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760791.512219:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.512231:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.512252:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760791.512281:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565563520:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.512508:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18224, 18224] 00000080:00200000:3.0:1789760791.512525:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.512531:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.512545:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74649600 i_blocks 145824 00000020:00200000:3.0:1789760791.512549:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.512558:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.512592:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74649600, count: 4096 00000080:00200000:3.0:1789760791.512599:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74649600, bytes: 4096 00000080:00200000:3.0:1789760791.512610:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.512617:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.512620:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74649600, 74653696) 0 0 00000080:00200000:3.0:1789760791.512624:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760791.512641:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18225, 18225] 00020000:00200000:3.0:1789760791.512881:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74649600 chunk: [74649600, 74653696) 74653696, 4096 00020000:00200000:3.0:1789760791.512886:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760791.512906:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18225, 18225] 00000008:00010000:3.0:1789760791.512924:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760791.512930:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760791.512941:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74649600, 74653696) append 0 append lockpos 0 00000080:00200000:3.0:1789760791.512954:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74649600=0x4731000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760791.513011:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760791.513014:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74653696 00000020:00010000:3.0:1789760791.513021:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760791.513025:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.513028:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.513032:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760791.513035:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760791.513044:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760791.513050:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760791.513067:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760791.513189:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 74649600<>74653696 00000100:00100000:2.0:1789760791.513379:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565563648:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.525190:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985533 gen 3 00000008:00010000:2.0:1789760791.525244:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760791.525253:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.525265:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.525355:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760791.525399:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565563648:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.525577:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18225, 18225] 00000080:00200000:3.0:1789760791.525684:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.525689:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.527843:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74653696 i_blocks 145832 00000020:00200000:2.0:1789760791.527859:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.527880:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.527918:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74653696, count: 4096 00000080:00200000:2.0:1789760791.527929:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74653696, bytes: 4096 00000080:00200000:2.0:1789760791.527944:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.527953:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.527955:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74653696, 74657792) 0 0 00000080:00200000:2.0:1789760791.527961:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.528246:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18226, 18226] 00020000:00200000:2.0:1789760791.528258:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74653696 chunk: [74653696, 74657792) 74657792, 4096 00020000:00200000:2.0:1789760791.528263:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.528286:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18226, 18226] 00000008:00010000:2.0:1789760791.528305:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.528311:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.528324:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74653696, 74657792) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.528340:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74653696=0x4732000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.528477:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.528480:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74657792 00000020:00010000:2.0:1789760791.528490:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.528493:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.528496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.528500:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.528502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.528510:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.528517:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.529097:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.529247:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 74653696<>74657792 00000100:00100000:3.0:1789760791.529321:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760791.529547:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565563776:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.538370:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985534 gen 3 00000008:00010000:2.0:1789760791.538433:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.538442:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.538454:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.538520:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.538551:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565563776:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.538819:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18226, 18226] 00000080:00200000:3.0:1789760791.538834:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.538838:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.538951:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74657792 i_blocks 145840 00000020:00200000:2.0:1789760791.538956:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.538965:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.539068:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74657792, count: 4096 00000080:00200000:2.0:1789760791.539077:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74657792, bytes: 4096 00000080:00200000:2.0:1789760791.539088:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.539097:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.539100:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74657792, 74661888) 0 0 00000080:00200000:2.0:1789760791.539105:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.539123:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18227, 18227] 00020000:00200000:2.0:1789760791.539200:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74657792 chunk: [74657792, 74661888) 74661888, 4096 00020000:00200000:2.0:1789760791.539205:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.539235:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18227, 18227] 00000008:00010000:2.0:1789760791.539254:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.539260:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.539272:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74657792, 74661888) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.539286:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74657792=0x4733000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.539336:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.539338:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74661888 00000020:00010000:2.0:1789760791.539354:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.539359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.539362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.539366:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.539369:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.539384:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.539392:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.539922:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.540199:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 74657792<>74661888 00000100:00100000:2.0:1789760791.540270:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760791.540292:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565563904:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.553073:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985535 gen 3 00000008:00010000:2.0:1789760791.553130:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.553139:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.553152:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.553179:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.553212:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565563904:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760791.553455:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18227, 18227] 00000080:00200000:1.0:1789760791.553474:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.553480:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.553497:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74661888 i_blocks 145848 00000020:00200000:1.0:1789760791.553501:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.553512:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.553549:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74661888, count: 4096 00000080:00200000:1.0:1789760791.553557:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74661888, bytes: 4096 00000080:00200000:1.0:1789760791.553569:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.553579:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.553582:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74661888, 74665984) 0 0 00000080:00200000:1.0:1789760791.553587:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.553604:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18228, 18228] 00020000:00200000:1.0:1789760791.553615:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74661888 chunk: [74661888, 74665984) 74665984, 4096 00020000:00200000:1.0:1789760791.553620:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.553640:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18228, 18228] 00000008:00010000:1.0:1789760791.553684:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.553691:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.553702:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74661888, 74665984) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.553717:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74661888=0x4734000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.553775:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.553778:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74665984 00000020:00010000:1.0:1789760791.553786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.553790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.553793:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.553796:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.553799:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.553806:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.553813:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.553831:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760791.553953:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74661888<>74665984 00000100:00100000:0.0:1789760791.554241:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564032:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.572837:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985536 gen 3 00000008:00010000:1.0:1789760791.572887:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760791.572897:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.572908:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.572934:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760791.572962:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564032:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.573619:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18228, 18228] 00000080:00200000:2.0:1789760791.573637:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.573677:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.573694:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74665984 i_blocks 145856 00000020:00200000:2.0:1789760791.573697:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.573707:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.573739:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74665984, count: 4096 00000080:00200000:2.0:1789760791.573746:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74665984, bytes: 4096 00000080:00200000:2.0:1789760791.573757:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.573765:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.573767:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74665984, 74670080) 0 0 00000080:00200000:2.0:1789760791.573771:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.573787:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18229, 18229] 00020000:00200000:2.0:1789760791.573795:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74665984 chunk: [74665984, 74670080) 74670080, 4096 00020000:00200000:2.0:1789760791.573799:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.573817:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18229, 18229] 00000008:00010000:2.0:1789760791.573834:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.573839:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.573850:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74665984, 74670080) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.573863:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74665984=0x4735000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.573916:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.573919:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74670080 00000020:00010000:2.0:1789760791.573927:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.573930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.573933:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.573936:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.573939:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.573947:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.573953:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.573970:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.574090:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 74665984<>74670080 00000100:00100000:3.0:1789760791.574316:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760791.574336:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565564160:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.604804:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985537 gen 3 00000008:00010000:2.0:1789760791.606422:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:2.0:1789760791.606437:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.606568:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.606600:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:2.0:1789760791.606691:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565564160:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760791.606827:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18229, 18229] 00000080:00200000:1.0:1789760791.606845:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.606851:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.606867:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74670080 i_blocks 145864 00000020:00200000:1.0:1789760791.606870:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.606880:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.606915:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74670080, count: 4096 00000080:00200000:1.0:1789760791.606923:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74670080, bytes: 4096 00000080:00200000:1.0:1789760791.606937:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.606946:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.606948:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74670080, 74674176) 0 0 00000080:00200000:1.0:1789760791.606952:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.606970:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18230, 18230] 00020000:00200000:1.0:1789760791.606978:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74670080 chunk: [74670080, 74674176) 74674176, 4096 00020000:00200000:1.0:1789760791.606983:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.607002:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18230, 18230] 00000008:00010000:1.0:1789760791.607021:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.607027:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.607039:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74670080, 74674176) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.607052:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74670080=0x4736000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.607110:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.607113:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74674176 00000020:00010000:1.0:1789760791.607121:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.607125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.607128:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.607132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.607260:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.607269:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.607277:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.607293:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760791.609712:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74670080<>74674176 00000100:00100000:0.0:1789760791.609828:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565564288:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.627842:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985538 gen 3 00000008:00010000:1.0:1789760791.627895:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760791.627903:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.627913:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.627934:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760791.627963:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565564288:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.628076:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18230, 18230] 00000080:00200000:0.0:1789760791.628094:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.628099:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.628115:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74674176 i_blocks 145872 00000020:00200000:0.0:1789760791.628119:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.628128:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.628164:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74674176, count: 4096 00000080:00200000:0.0:1789760791.628325:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74674176, bytes: 4096 00000080:00200000:0.0:1789760791.628338:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.628346:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.628349:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74674176, 74678272) 0 0 00000080:00200000:0.0:1789760791.628353:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.628370:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18231, 18231] 00020000:00200000:0.0:1789760791.628379:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74674176 chunk: [74674176, 74678272) 74678272, 4096 00020000:00200000:0.0:1789760791.628383:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.628403:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18231, 18231] 00000008:00010000:0.0:1789760791.628433:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.628439:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.628451:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74674176, 74678272) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.628465:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74674176=0x4737000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.628799:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.628809:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74678272 00000020:00010000:0.0:1789760791.628818:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.628822:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.628824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.628828:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.628830:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.628839:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.628846:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.628866:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760791.628988:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 74674176<>74678272 00000100:00100000:1.0:1789760791.629388:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564416:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.645483:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985539 gen 3 00000008:00010000:1.0:1789760791.645536:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760791.645545:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.645556:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.645580:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760791.645607:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564416:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.646166:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18231, 18231] 00000080:00200000:0.0:1789760791.646192:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.646198:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.646215:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74678272 i_blocks 145880 00000020:00200000:0.0:1789760791.646229:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.646240:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.646275:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74678272, count: 4096 00000080:00200000:0.0:1789760791.646282:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74678272, bytes: 4096 00000080:00200000:0.0:1789760791.646295:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.646304:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.646307:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74678272, 74682368) 0 0 00000080:00200000:0.0:1789760791.646311:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.646328:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18232, 18232] 00020000:00200000:0.0:1789760791.646337:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74678272 chunk: [74678272, 74682368) 74682368, 4096 00020000:00200000:0.0:1789760791.646342:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.646642:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18232, 18232] 00000008:00010000:0.0:1789760791.646691:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.646698:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.646712:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74678272, 74682368) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.646727:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74678272=0x4738000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.646797:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.646800:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74682368 00000020:00010000:0.0:1789760791.646807:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.646811:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.646813:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.646817:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.646820:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.646829:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.646835:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.646856:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dcc0 00000008:00100000:0.0:1789760791.647094:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 74678272<>74682368 00000100:00100000:1.0:1789760791.647316:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760791.647341:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564544:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.662152:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985540 gen 3 00000008:00010000:1.0:1789760791.662206:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760791.662215:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.662238:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.662259:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760791.662290:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564544:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.662419:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18232, 18232] 00000080:00200000:0.0:1789760791.662437:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.662443:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.662459:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74682368 i_blocks 145888 00000020:00200000:0.0:1789760791.662463:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.662474:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.662510:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74682368, count: 4096 00000080:00200000:0.0:1789760791.662517:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74682368, bytes: 4096 00000080:00200000:0.0:1789760791.662528:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.662548:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.662551:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74682368, 74686464) 0 0 00000080:00200000:0.0:1789760791.662555:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.662575:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18233, 18233] 00020000:00200000:0.0:1789760791.662584:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74682368 chunk: [74682368, 74686464) 74686464, 4096 00020000:00200000:0.0:1789760791.662589:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.662612:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18233, 18233] 00000008:00010000:0.0:1789760791.662632:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.662640:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.662687:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74682368, 74686464) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.662703:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74682368=0x4739000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.662774:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.662777:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74686464 00000020:00010000:0.0:1789760791.662786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.662790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.662793:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.662796:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.662799:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.662806:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.662812:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.663023:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760791.663168:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 74682368<>74686464 00000100:00100000:1.0:1789760791.663398:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564672:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.675257:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985541 gen 3 00000008:00010000:1.0:1789760791.675319:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760791.675329:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.675342:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.675366:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:1.0:1789760791.675396:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564672:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.675529:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18233, 18233] 00000080:00200000:0.0:1789760791.675547:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.675553:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.675571:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74686464 i_blocks 145896 00000020:00200000:0.0:1789760791.675575:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.675585:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.675623:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74686464, count: 4096 00000080:00200000:0.0:1789760791.675631:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74686464, bytes: 4096 00000080:00200000:0.0:1789760791.675642:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.675698:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.675700:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74686464, 74690560) 0 0 00000080:00200000:0.0:1789760791.675704:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.675720:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18234, 18234] 00020000:00200000:0.0:1789760791.675730:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74686464 chunk: [74686464, 74690560) 74690560, 4096 00020000:00200000:0.0:1789760791.675735:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.675756:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18234, 18234] 00000008:00010000:0.0:1789760791.675774:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.675781:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.675794:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74686464, 74690560) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.675810:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74686464=0x473a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.675867:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.675870:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74690560 00000020:00010000:0.0:1789760791.675879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.675882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.675886:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.675890:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.675893:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.675900:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.675907:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.675925:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760791.676052:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74686464<>74690560 00000100:00100000:1.0:1789760791.676152:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760791.676170:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564800:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.685319:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985542 gen 3 00000008:00010000:1.0:1789760791.685371:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760791.685379:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.685390:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.685421:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760791.685451:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564800:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.685633:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18234, 18234] 00000080:00200000:0.0:1789760791.685686:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.685693:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.685709:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74690560 i_blocks 145904 00000020:00200000:0.0:1789760791.685714:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.685726:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.685758:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74690560, count: 4096 00000080:00200000:0.0:1789760791.685766:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74690560, bytes: 4096 00000080:00200000:0.0:1789760791.685777:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.685786:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.685788:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74690560, 74694656) 0 0 00000080:00200000:0.0:1789760791.685793:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.685812:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18235, 18235] 00020000:00200000:0.0:1789760791.685821:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74690560 chunk: [74690560, 74694656) 74694656, 4096 00020000:00200000:0.0:1789760791.685825:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.685846:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18235, 18235] 00000008:00010000:0.0:1789760791.685863:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.685870:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.685882:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74690560, 74694656) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.685896:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74690560=0x473b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.685955:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.685958:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74694656 00000020:00010000:0.0:1789760791.685966:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.685970:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.685973:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.685977:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.685980:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.685987:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.685994:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.686011:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760791.686133:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 74690560<>74694656 00000100:00100000:1.0:1789760791.686746:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564928:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.694865:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985543 gen 3 00000008:00010000:1.0:1789760791.694932:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760791.694946:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.694958:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.694977:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760791.695007:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565564928:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.695119:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18235, 18235] 00000080:00200000:0.0:1789760791.695140:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.695146:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.695165:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74694656 i_blocks 145912 00000020:00200000:0.0:1789760791.695168:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.695180:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.695228:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74694656, count: 4096 00000080:00200000:0.0:1789760791.695236:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74694656, bytes: 4096 00000080:00200000:0.0:1789760791.695250:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.695259:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.695262:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74694656, 74698752) 0 0 00000080:00200000:0.0:1789760791.695266:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.695289:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18236, 18236] 00020000:00200000:0.0:1789760791.695298:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74694656 chunk: [74694656, 74698752) 74698752, 4096 00020000:00200000:0.0:1789760791.695303:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.695324:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18236, 18236] 00000008:00010000:0.0:1789760791.695341:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.695348:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.695359:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74694656, 74698752) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.695373:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74694656=0x473c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.695441:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.695445:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74698752 00000020:00010000:0.0:1789760791.695453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.695456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.695459:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.695462:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.695465:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.695472:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.695478:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.695494:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760791.695618:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 74694656<>74698752 00000100:00100000:1.0:1789760791.695845:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760791.695864:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565565056:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.712805:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985544 gen 3 00000008:00010000:1.0:1789760791.712858:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760791.712867:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.712878:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.712901:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:1.0:1789760791.712937:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565565056:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.713566:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18236, 18236] 00000080:00200000:0.0:1789760791.713584:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.713590:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.713607:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74698752 i_blocks 145920 00000020:00200000:0.0:1789760791.713610:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.713622:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.713703:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74698752, count: 4096 00000080:00200000:0.0:1789760791.713710:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74698752, bytes: 4096 00000080:00200000:0.0:1789760791.713721:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.713730:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.713732:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74698752, 74702848) 0 0 00000080:00200000:0.0:1789760791.713736:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.713761:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18237, 18237] 00020000:00200000:0.0:1789760791.713769:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74698752 chunk: [74698752, 74702848) 74702848, 4096 00020000:00200000:0.0:1789760791.713774:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.713794:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18237, 18237] 00000008:00010000:0.0:1789760791.713812:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.713818:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.713830:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74698752, 74702848) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.713844:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74698752=0x473d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.713904:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.713907:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74702848 00000020:00010000:0.0:1789760791.713916:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.713919:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.713922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.713925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.713928:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.713935:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.713942:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.713961:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dcc0 00000008:00100000:0.0:1789760791.714092:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 74698752<>74702848 00000100:00100000:1.0:1789760791.715622:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565565184:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.729815:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985545 gen 3 00000008:00010000:1.0:1789760791.729867:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760791.729876:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.729887:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.729918:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760791.729949:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565565184:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.730782:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18237, 18237] 00000080:00200000:0.0:1789760791.730801:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.730808:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.730825:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74702848 i_blocks 145928 00000020:00200000:0.0:1789760791.730828:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.730840:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.730873:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74702848, count: 4096 00000080:00200000:0.0:1789760791.730880:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74702848, bytes: 4096 00000080:00200000:0.0:1789760791.730891:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.730899:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.730901:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74702848, 74706944) 0 0 00000080:00200000:0.0:1789760791.730906:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.730922:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18238, 18238] 00020000:00200000:0.0:1789760791.730931:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74702848 chunk: [74702848, 74706944) 74706944, 4096 00020000:00200000:0.0:1789760791.730936:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.730955:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18238, 18238] 00000008:00010000:0.0:1789760791.730972:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.730977:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.730989:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74702848, 74706944) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.731004:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74702848=0x473e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.731068:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.731071:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74706944 00000020:00010000:0.0:1789760791.731079:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.731082:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.731085:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.731089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.731092:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.731100:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.731107:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.731127:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760791.731249:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 74702848<>74706944 00000100:00100000:1.0:1789760791.731502:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760791.731525:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565565312:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.746580:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985546 gen 3 00000008:00010000:1.0:1789760791.746638:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760791.746745:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.747906:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.747998:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760791.748035:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565565312:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.750990:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18238, 18238] 00000080:00200000:2.0:1789760791.751020:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.751027:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.751072:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74706944 i_blocks 145936 00000020:00200000:2.0:1789760791.751078:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.751099:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.751132:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74706944, count: 4096 00000080:00200000:2.0:1789760791.751141:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74706944, bytes: 4096 00000080:00200000:2.0:1789760791.751156:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.751166:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.751169:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74706944, 74711040) 0 0 00000080:00200000:2.0:1789760791.751173:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.751191:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18239, 18239] 00020000:00200000:2.0:1789760791.751200:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74706944 chunk: [74706944, 74711040) 74711040, 4096 00020000:00200000:2.0:1789760791.751205:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.751238:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18239, 18239] 00000008:00010000:2.0:1789760791.751259:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.751266:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.751279:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74706944, 74711040) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.751294:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74706944=0x473f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.751354:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.751357:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74711040 00000020:00010000:2.0:1789760791.751366:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.751370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.751372:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.751376:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.751380:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.751388:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.751398:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.751425:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f0c0 00000008:00100000:2.0:1789760791.751556:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 74706944<>74711040 00000100:00100000:3.0:1789760791.751889:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565565440:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.763583:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985547 gen 3 00000008:00010000:3.0:1789760791.763684:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f0c0 00000020:00200000:3.0:1789760791.763694:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760791.763705:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760791.763729:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f0c0 00000100:00100000:3.0:1789760791.763770:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565565440:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.763969:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18239, 18239] 00000080:00200000:2.0:1789760791.763987:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.763993:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.764007:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74711040 i_blocks 145944 00000020:00200000:2.0:1789760791.764011:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.764020:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.764051:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74711040, count: 4096 00000080:00200000:2.0:1789760791.764058:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74711040, bytes: 4096 00000080:00200000:2.0:1789760791.764067:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.764075:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.764077:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74711040, 74715136) 0 0 00000080:00200000:2.0:1789760791.764081:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.764097:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18240, 18240] 00020000:00200000:2.0:1789760791.764106:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74711040 chunk: [74711040, 74715136) 74715136, 4096 00020000:00200000:2.0:1789760791.764110:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.764129:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18240, 18240] 00000008:00010000:2.0:1789760791.764146:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.764152:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.764165:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74711040, 74715136) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.764178:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74711040=0x4740000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.764291:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.764294:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74715136 00000020:00010000:2.0:1789760791.764304:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.764307:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.764311:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.764314:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.764317:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.764326:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.764335:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.764425:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760791.764545:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 74711040<>74715136 00000100:00100000:3.0:1789760791.764709:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760791.764733:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565565568:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.772389:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985548 gen 3 00000008:00010000:3.0:1789760791.772450:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760791.772459:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760791.772470:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760791.772505:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760791.772536:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565565568:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.772858:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18240, 18240] 00000080:00200000:2.0:1789760791.772876:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.772882:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.772899:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74715136 i_blocks 145952 00000020:00200000:2.0:1789760791.772902:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.772912:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.772949:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74715136, count: 4096 00000080:00200000:2.0:1789760791.772958:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74715136, bytes: 4096 00000080:00200000:2.0:1789760791.772968:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.772977:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.772980:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74715136, 74719232) 0 0 00000080:00200000:2.0:1789760791.772985:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.773003:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18241, 18241] 00020000:00200000:2.0:1789760791.773012:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74715136 chunk: [74715136, 74719232) 74719232, 4096 00020000:00200000:2.0:1789760791.773017:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.773036:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18241, 18241] 00000008:00010000:2.0:1789760791.773053:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.773058:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.773069:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74715136, 74719232) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.773081:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74715136=0x4741000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.773142:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.773145:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74719232 00000020:00010000:2.0:1789760791.773154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.773157:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.773160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354ecf8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.773164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.773166:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.773175:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.773184:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.773202:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760791.774068:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 74715136<>74719232 00000100:00100000:2.0:1789760791.774129:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760791.774152:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565565696:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.787521:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985549 gen 3 00000008:00010000:2.0:1789760791.787573:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760791.787582:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.787593:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.787715:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000080:00200000:3.0:1789760791.787726:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18241, 18241] 00000100:00100000:2.0:1789760791.787747:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565565696:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.787843:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.787853:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.788066:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74719232 i_blocks 145960 00000020:00200000:2.0:1789760791.788070:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.788079:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.788110:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74719232, count: 4096 00000080:00200000:2.0:1789760791.788115:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74719232, bytes: 4096 00000080:00200000:2.0:1789760791.788124:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.788131:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.788133:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74719232, 74723328) 0 0 00000080:00200000:2.0:1789760791.788136:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.788155:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18242, 18242] 00020000:00200000:2.0:1789760791.788163:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74719232 chunk: [74719232, 74723328) 74723328, 4096 00020000:00200000:2.0:1789760791.788168:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.788186:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18242, 18242] 00000008:00010000:2.0:1789760791.788205:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.788210:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.788227:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74719232, 74723328) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.788241:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74719232=0x4742000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.788300:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.788304:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74723328 00000020:00010000:2.0:1789760791.788312:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.788316:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.788320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354ebd8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.788324:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0d58 (ffff93f84354e7e8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.788327:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e7e8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.788334:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.788342:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.788452:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760791.788876:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 74719232<>74723328 00000100:00100000:3.0:1789760791.788928:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760791.789064:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565565824:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.797278:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985550 gen 3 00000008:00010000:2.0:1789760791.797328:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760791.797338:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.797482:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.797608:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760791.797677:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565565824:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.797848:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18242, 18242] 00000080:00200000:3.0:1789760791.797901:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.797908:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.797925:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74723328 i_blocks 145968 00000020:00200000:3.0:1789760791.797930:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.797940:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.797988:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74723328, count: 4096 00000080:00200000:3.0:1789760791.797996:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74723328, bytes: 4096 00000080:00200000:3.0:1789760791.798008:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.798017:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.798019:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74723328, 74727424) 0 0 00000080:00200000:3.0:1789760791.798023:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760791.798039:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18243, 18243] 00020000:00200000:3.0:1789760791.798049:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74723328 chunk: [74723328, 74727424) 74727424, 4096 00020000:00200000:3.0:1789760791.798054:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760791.798073:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18243, 18243] 00000008:00010000:3.0:1789760791.798092:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760791.798098:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760791.798111:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74723328, 74727424) append 0 append lockpos 0 00000080:00200000:3.0:1789760791.798124:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74723328=0x4743000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760791.798187:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760791.798190:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74727424 00000020:00010000:3.0:1789760791.798199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:3.0:1789760791.798202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.798205:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.798209:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760791.798212:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760791.798221:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760791.798229:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760791.798247:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760791.798599:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 74723328<>74727424 00000100:00100000:2.0:1789760791.798674:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565565952:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.806479:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985551 gen 3 00000008:00010000:2.0:1789760791.806587:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760791.806595:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.806607:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.806894:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:2.0:1789760791.806926:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565565952:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.807012:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18243, 18243] 00000080:00200000:0.0:1789760791.807029:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.807035:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.807172:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74727424 i_blocks 145976 00000020:00200000:1.0:1789760791.807178:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.807189:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.807228:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74727424, count: 4096 00000080:00200000:1.0:1789760791.807236:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74727424, bytes: 4096 00000080:00200000:1.0:1789760791.807246:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.807254:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.807256:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74727424, 74731520) 0 0 00000080:00200000:1.0:1789760791.807261:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.807278:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18244, 18244] 00020000:00200000:1.0:1789760791.807287:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74727424 chunk: [74727424, 74731520) 74731520, 4096 00020000:00200000:1.0:1789760791.807291:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.807311:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18244, 18244] 00000008:00010000:1.0:1789760791.807328:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.807335:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.807417:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74727424, 74731520) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.807432:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74727424=0x4744000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.807542:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.807545:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74731520 00000020:00010000:1.0:1789760791.807553:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.807557:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.807560:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.807563:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.807566:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.807573:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.807579:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.807595:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760791.807741:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74727424<>74731520 00000100:00100000:1.0:1789760791.807894:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760791.807915:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565566080:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.817545:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985552 gen 3 00000008:00010000:1.0:1789760791.817596:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760791.817605:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.817616:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.817642:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760791.817702:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565566080:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.817822:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18244, 18244] 00000080:00200000:2.0:1789760791.817838:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.817845:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.817860:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74731520 i_blocks 145984 00000020:00200000:2.0:1789760791.817864:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.817874:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.817907:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74731520, count: 4096 00000080:00200000:2.0:1789760791.817914:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74731520, bytes: 4096 00000080:00200000:2.0:1789760791.817924:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.817932:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.817934:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74731520, 74735616) 0 0 00000080:00200000:2.0:1789760791.817938:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.817954:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18245, 18245] 00020000:00200000:2.0:1789760791.817962:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74731520 chunk: [74731520, 74735616) 74735616, 4096 00020000:00200000:2.0:1789760791.817966:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.817986:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18245, 18245] 00000008:00010000:2.0:1789760791.818004:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.818010:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.818020:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74731520, 74735616) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.818033:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74731520=0x4745000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.818085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.818087:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74735616 00000020:00010000:2.0:1789760791.818095:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.818098:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.818102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.818105:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.818108:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.818114:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.818120:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.818135:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760791.818248:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 74731520<>74735616 00000100:00100000:3.0:1789760791.818316:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565566208:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.825441:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985553 gen 3 00000008:00010000:2.0:1789760791.825886:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760791.825895:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.825907:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.825933:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760791.825966:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565566208:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760791.826136:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18245, 18245] 00000080:00200000:1.0:1789760791.826162:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760791.826168:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760791.826185:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74735616 i_blocks 145992 00000020:00200000:1.0:1789760791.826190:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760791.826200:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760791.826235:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74735616, count: 4096 00000080:00200000:1.0:1789760791.826243:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74735616, bytes: 4096 00000080:00200000:1.0:1789760791.826254:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760791.826263:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760791.826266:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74735616, 74739712) 0 0 00000080:00200000:1.0:1789760791.826270:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760791.826287:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18246, 18246] 00020000:00200000:1.0:1789760791.826296:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74735616 chunk: [74735616, 74739712) 74739712, 4096 00020000:00200000:1.0:1789760791.826301:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760791.826321:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18246, 18246] 00000008:00010000:1.0:1789760791.826340:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760791.826476:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760791.826578:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74735616, 74739712) append 0 append lockpos 0 00000080:00200000:1.0:1789760791.826593:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74735616=0x4746000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760791.826679:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760791.826682:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74739712 00000020:00010000:1.0:1789760791.826690:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:1.0:1789760791.826694:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.826697:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760791.826701:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760791.826704:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760791.826712:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760791.826719:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760791.826740:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760791.827104:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74735616<>74739712 00000100:00100000:0.0:1789760791.827197:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565566336:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760791.839772:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985554 gen 3 00000008:00010000:1.0:1789760791.839822:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760791.839831:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760791.839843:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760791.839874:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760791.839905:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565566336:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.841687:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18246, 18246] 00000080:00200000:2.0:1789760791.841712:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.841719:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.841736:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74739712 i_blocks 146000 00000020:00200000:2.0:1789760791.841741:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.841756:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.841790:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74739712, count: 4096 00000080:00200000:2.0:1789760791.841799:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74739712, bytes: 4096 00000080:00200000:2.0:1789760791.841812:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.841821:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.841824:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74739712, 74743808) 0 0 00000080:00200000:2.0:1789760791.841829:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.841845:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18247, 18247] 00020000:00200000:2.0:1789760791.841854:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74739712 chunk: [74739712, 74743808) 74743808, 4096 00020000:00200000:2.0:1789760791.841858:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.841881:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18247, 18247] 00000008:00010000:2.0:1789760791.841899:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.841905:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.841917:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74739712, 74743808) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.841932:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74739712=0x4747000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.841999:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.842002:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74743808 00000020:00010000:2.0:1789760791.842011:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.842015:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.842019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.842023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.842026:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.842034:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.842042:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.842416:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760791.842630:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 74739712<>74743808 00000100:00100000:2.0:1789760791.842748:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760791.842773:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565566464:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.859777:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985555 gen 3 00000008:00010000:2.0:1789760791.859831:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760791.859840:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.859851:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.859876:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760791.859907:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565566464:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.860027:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18247, 18247] 00000080:00200000:3.0:1789760791.860043:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.860065:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.860082:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74743808 i_blocks 146008 00000020:00200000:3.0:1789760791.860086:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.860097:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.860132:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74743808, count: 4096 00000080:00200000:3.0:1789760791.860139:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74743808, bytes: 4096 00000080:00200000:3.0:1789760791.860162:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.860172:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.860175:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74743808, 74747904) 0 0 00000080:00200000:3.0:1789760791.860179:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760791.860196:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18248, 18248] 00020000:00200000:3.0:1789760791.860206:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74743808 chunk: [74743808, 74747904) 74747904, 4096 00020000:00200000:3.0:1789760791.860211:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760791.860232:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18248, 18248] 00000008:00010000:3.0:1789760791.860250:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760791.860257:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760791.860269:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74743808, 74747904) append 0 append lockpos 0 00000080:00200000:3.0:1789760791.860284:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74743808=0x4748000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760791.860344:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760791.860347:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74747904 00000020:00010000:3.0:1789760791.860356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:3.0:1789760791.860359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.860363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.860366:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760791.860371:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760791.860380:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760791.860388:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760791.860405:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760791.860558:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 74743808<>74747904 00000100:00100000:3.0:1789760791.860607:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760791.860625:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565566592:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.878996:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985556 gen 3 00000008:00010000:3.0:1789760791.879051:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:3.0:1789760791.879060:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760791.879072:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760791.879099:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:3.0:1789760791.879130:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565566592:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.879231:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18248, 18248] 00000080:00200000:2.0:1789760791.879248:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.879254:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.879271:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74747904 i_blocks 146016 00000020:00200000:2.0:1789760791.879274:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.879285:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.879320:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74747904, count: 4096 00000080:00200000:2.0:1789760791.879328:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74747904, bytes: 4096 00000080:00200000:2.0:1789760791.879338:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.879347:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.879349:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74747904, 74752000) 0 0 00000080:00200000:2.0:1789760791.879354:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.879371:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18249, 18249] 00020000:00200000:2.0:1789760791.879384:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74747904 chunk: [74747904, 74752000) 74752000, 4096 00020000:00200000:2.0:1789760791.879389:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.879417:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18249, 18249] 00000008:00010000:2.0:1789760791.879436:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.879442:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.879454:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74747904, 74752000) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.879469:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74747904=0x4749000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.879529:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.879532:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74752000 00000020:00010000:2.0:1789760791.879598:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.879602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.879605:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.879608:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.879611:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.879618:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.879625:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.879663:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760791.879783:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 74747904<>74752000 00000100:00100000:2.0:1789760791.879830:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760791.879845:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565566720:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.891351:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985557 gen 3 00000008:00010000:2.0:1789760791.891401:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760791.891420:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.891430:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.891454:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760791.891484:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565566720:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.891537:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18249, 18249] 00000080:00200000:3.0:1789760791.891554:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.891560:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.891575:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74752000 i_blocks 146024 00000020:00200000:3.0:1789760791.891578:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.891588:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.891624:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74752000, count: 4096 00000080:00200000:3.0:1789760791.891632:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74752000, bytes: 4096 00000080:00200000:3.0:1789760791.891686:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.891696:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.891699:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74752000, 74756096) 0 0 00000080:00200000:3.0:1789760791.891704:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760791.891723:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18250, 18250] 00020000:00200000:3.0:1789760791.891734:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74752000 chunk: [74752000, 74756096) 74756096, 4096 00020000:00200000:3.0:1789760791.891739:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760791.891770:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18250, 18250] 00000008:00010000:2.0:1789760791.891820:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.891827:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.891839:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74752000, 74756096) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.891855:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74752000=0x474a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.891907:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.891910:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74756096 00000020:00010000:2.0:1789760791.891919:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.891922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.891925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.891930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.891933:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.891941:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.891949:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.893097:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760791.893248:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 74752000<>74756096 00000100:00100000:3.0:1789760791.894131:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760791.894167:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565566848:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.910425:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985558 gen 3 00000008:00010000:3.0:1789760791.910478:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760791.910487:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760791.910499:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760791.910527:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760791.910562:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565566848:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.911433:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18250, 18250] 00000080:00200000:2.0:1789760791.911453:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.911459:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.911476:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74756096 i_blocks 146032 00000020:00200000:2.0:1789760791.911479:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.911492:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.911526:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74756096, count: 4096 00000080:00200000:2.0:1789760791.911534:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74756096, bytes: 4096 00000080:00200000:2.0:1789760791.911548:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.911557:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.911560:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74756096, 74760192) 0 0 00000080:00200000:2.0:1789760791.911565:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.911581:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18251, 18251] 00020000:00200000:2.0:1789760791.911590:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74756096 chunk: [74756096, 74760192) 74760192, 4096 00020000:00200000:2.0:1789760791.911595:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.911616:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18251, 18251] 00000008:00010000:2.0:1789760791.911636:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.911701:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.911714:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74756096, 74760192) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.911728:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74756096=0x474b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.911785:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.911789:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74760192 00000020:00010000:2.0:1789760791.911797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.911801:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.911804:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.911809:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.911812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.911832:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.911840:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.911859:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760791.911984:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 74756096<>74760192 00000100:00100000:3.0:1789760791.913862:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565566976:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760791.926010:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985559 gen 3 00000008:00010000:3.0:1789760791.926063:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760791.926072:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760791.926082:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760791.926108:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760791.926141:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565566976:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.926292:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18251, 18251] 00000080:00200000:2.0:1789760791.926309:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.926316:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.926332:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74760192 i_blocks 146040 00000020:00200000:2.0:1789760791.926336:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.926346:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.926381:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74760192, count: 4096 00000080:00200000:2.0:1789760791.926389:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74760192, bytes: 4096 00000080:00200000:2.0:1789760791.926400:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.926417:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.926420:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74760192, 74764288) 0 0 00000080:00200000:2.0:1789760791.926425:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.926441:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18252, 18252] 00020000:00200000:2.0:1789760791.926450:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74760192 chunk: [74760192, 74764288) 74764288, 4096 00020000:00200000:2.0:1789760791.926455:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.926475:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18252, 18252] 00000008:00010000:2.0:1789760791.926492:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.926498:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.926510:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74760192, 74764288) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.926525:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74760192=0x474c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.926583:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.926586:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74764288 00000020:00010000:2.0:1789760791.926595:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.926599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.926602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.926606:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.926609:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.926617:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.926624:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.926642:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760791.926814:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74760192<>74764288 00000100:00100000:2.0:1789760791.927003:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760791.927023:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565567104:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.945769:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985560 gen 3 00000008:00010000:2.0:1789760791.945824:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760791.945834:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.945845:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.945872:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760791.945905:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565567104:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.946398:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18252, 18252] 00000080:00200000:3.0:1789760791.946425:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.946432:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.946449:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74764288 i_blocks 146048 00000020:00200000:3.0:1789760791.946453:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.946464:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.946501:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74764288, count: 4096 00000080:00200000:3.0:1789760791.946509:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74764288, bytes: 4096 00000080:00200000:3.0:1789760791.946520:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.946528:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.946531:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74764288, 74768384) 0 0 00000080:00200000:3.0:1789760791.946536:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760791.946553:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18253, 18253] 00020000:00200000:3.0:1789760791.946563:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74764288 chunk: [74764288, 74768384) 74768384, 4096 00020000:00200000:3.0:1789760791.946570:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760791.946591:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18253, 18253] 00000008:00010000:3.0:1789760791.946611:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760791.946618:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760791.946630:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74764288, 74768384) append 0 append lockpos 0 00000080:00200000:3.0:1789760791.946684:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74764288=0x474d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760791.946750:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760791.947247:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74768384 00000020:00010000:3.0:1789760791.947257:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:3.0:1789760791.947261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.947264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760791.947267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760791.947270:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760791.947280:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760791.947287:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760791.947310:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760791.947569:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 74764288<>74768384 00000100:00100000:2.0:1789760791.947676:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565567232:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.964274:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985561 gen 3 00000008:00010000:2.0:1789760791.964325:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760791.964332:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.964341:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.964363:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760791.964387:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565567232:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760791.964574:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18253, 18253] 00000080:00200000:3.0:1789760791.964593:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760791.964599:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760791.964614:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74768384 i_blocks 146056 00000020:00200000:3.0:1789760791.964617:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760791.964627:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760791.964707:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74768384, count: 4096 00000080:00200000:3.0:1789760791.964715:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74768384, bytes: 4096 00000080:00200000:3.0:1789760791.964725:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760791.964733:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760791.964736:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74768384, 74772480) 0 0 00000080:00200000:3.0:1789760791.964740:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760791.964771:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18254, 18254] 00020000:00200000:3.0:1789760791.964780:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74768384 chunk: [74768384, 74772480) 74772480, 4096 00020000:00200000:3.0:1789760791.964786:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760791.964807:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18254, 18254] 00000008:00010000:3.0:1789760791.964825:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760791.964830:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e6b0, osc lock ffff93f8457e7300, flags 0x0 00000080:00200000:3.0:1789760791.964842:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74768384, 74772480) append 0 append lockpos 0 00000080:00200000:3.0:1789760791.964856:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74768384=0x474e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.965163:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.965168:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74772480 00000020:00010000:0.0:1789760791.965178:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.965182:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.965185:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e6b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.965188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.965192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e6b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.965201:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.965208:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.965225:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760791.967817:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 74768384<>74772480 00000100:00100000:1.0:1789760791.967917:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565567360:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760791.979152:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985562 gen 3 00000008:00010000:0.0:1789760791.979204:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:0.0:1789760791.979214:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760791.979226:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760791.979260:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:0.0:1789760791.979291:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565567360:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760791.980733:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18254, 18254] 00000080:00200000:2.0:1789760791.980757:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760791.980763:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760791.980782:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74772480 i_blocks 146064 00000020:00200000:2.0:1789760791.980786:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760791.980800:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760791.980847:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74772480, count: 4096 00000080:00200000:2.0:1789760791.980856:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74772480, bytes: 4096 00000080:00200000:2.0:1789760791.980870:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760791.980879:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760791.980882:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74772480, 74776576) 0 0 00000080:00200000:2.0:1789760791.980886:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760791.980905:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18255, 18255] 00020000:00200000:2.0:1789760791.980914:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74772480 chunk: [74772480, 74776576) 74776576, 4096 00020000:00200000:2.0:1789760791.980919:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760791.980941:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18255, 18255] 00000008:00010000:2.0:1789760791.980959:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760791.980966:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760791.980978:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74772480, 74776576) append 0 append lockpos 0 00000080:00200000:2.0:1789760791.980991:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74772480=0x474f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760791.981059:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760791.981062:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74776576 00000020:00010000:2.0:1789760791.981070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760791.981073:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.981075:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760791.981078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760791.981081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760791.981089:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760791.981096:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760791.981116:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760791.981316:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74772480<>74776576 00000100:00100000:3.0:1789760791.981418:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565567488:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760791.993674:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985563 gen 3 00000008:00010000:2.0:1789760791.993726:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760791.993734:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760791.993744:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760791.993838:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760791.993870:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565567488:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760791.997028:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18255, 18255] 00000080:00200000:0.0:1789760791.997057:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760791.997063:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760791.997081:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74776576 i_blocks 146072 00000020:00200000:0.0:1789760791.997087:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760791.997105:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760791.997139:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74776576, count: 4096 00000080:00200000:0.0:1789760791.997148:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74776576, bytes: 4096 00000080:00200000:0.0:1789760791.997163:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760791.997172:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760791.997175:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74776576, 74780672) 0 0 00000080:00200000:0.0:1789760791.997179:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760791.997197:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18256, 18256] 00020000:00200000:0.0:1789760791.997206:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74776576 chunk: [74776576, 74780672) 74780672, 4096 00020000:00200000:0.0:1789760791.997212:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760791.997347:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18256, 18256] 00000008:00010000:0.0:1789760791.997368:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760791.997375:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760791.997388:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74776576, 74780672) append 0 append lockpos 0 00000080:00200000:0.0:1789760791.997402:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74776576=0x4750000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760791.997488:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760791.997491:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74780672 00000020:00010000:0.0:1789760791.997500:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760791.997504:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.997507:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760791.997510:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760791.997513:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760791.997522:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760791.997529:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760791.997560:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760791.997718:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 74776576<>74780672 00000100:00100000:1.0:1789760791.997878:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565567616:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760792.008343:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985564 gen 3 00000008:00010000:0.0:1789760792.008507:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:0.0:1789760792.008521:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760792.008532:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760792.008556:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:0.0:1789760792.008587:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565567616:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.008799:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18256, 18256] 00000080:00200000:2.0:1789760792.008818:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.008824:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.008850:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74780672 i_blocks 146080 00000020:00200000:2.0:1789760792.008855:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.008867:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.008903:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74780672, count: 4096 00000080:00200000:2.0:1789760792.008911:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74780672, bytes: 4096 00000080:00200000:2.0:1789760792.008922:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.008930:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.008932:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74780672, 74784768) 0 0 00000080:00200000:2.0:1789760792.008937:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.008955:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18257, 18257] 00020000:00200000:2.0:1789760792.008963:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74780672 chunk: [74780672, 74784768) 74784768, 4096 00020000:00200000:2.0:1789760792.008968:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.008990:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18257, 18257] 00000008:00010000:2.0:1789760792.009009:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.009015:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.009027:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74780672, 74784768) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.009043:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74780672=0x4751000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.009099:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.009101:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74784768 00000020:00010000:2.0:1789760792.009109:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.009113:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.009116:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.009119:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.009122:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.009130:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.009137:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.009153:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760792.009277:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74780672<>74784768 00000100:00100000:3.0:1789760792.009369:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565567744:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.021010:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985565 gen 3 00000008:00010000:2.0:1789760792.021062:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760792.021070:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.021081:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.021104:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760792.021134:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565567744:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.021309:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18257, 18257] 00000080:00200000:0.0:1789760792.021327:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.021333:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.021348:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74784768 i_blocks 146088 00000020:00200000:0.0:1789760792.021351:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.021362:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.021395:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74784768, count: 4096 00000080:00200000:0.0:1789760792.021402:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74784768, bytes: 4096 00000080:00200000:0.0:1789760792.021423:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.021432:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.021434:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74784768, 74788864) 0 0 00000080:00200000:0.0:1789760792.021438:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.021455:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18258, 18258] 00020000:00200000:0.0:1789760792.021463:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74784768 chunk: [74784768, 74788864) 74788864, 4096 00020000:00200000:0.0:1789760792.021468:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.021609:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18258, 18258] 00000008:00010000:0.0:1789760792.021629:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.021635:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.021684:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74784768, 74788864) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.021698:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74784768=0x4752000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.022066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.022070:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74788864 00000020:00010000:0.0:1789760792.022079:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.022083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.022085:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.022088:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.022091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.022099:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.022107:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.022276:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760792.022402:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 74784768<>74788864 00000100:00100000:1.0:1789760792.022499:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565567872:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760792.028272:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565568128:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1789760792.028393:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1789760792.028400:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:af634c28-3647-4f73-bb24-40d7878858fb:2358:1876699565568000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1789760792.028441:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6a301392-4ca2-4c43-8653-61d4c47997a9:2358:1876699565568256:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760792.041472:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:af634c28-3647-4f73-bb24-40d7878858fb:2358:1876699565568000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760792.042011:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6a301392-4ca2-4c43-8653-61d4c47997a9:2358:1876699565568256:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760792.042029:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565568128:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760792.042044:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985566 gen 3 00000008:00010000:1.0:1789760792.042080:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760792.042088:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.042099:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.042129:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760792.042156:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565567872:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.042344:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18258, 18258] 00000080:00200000:0.0:1789760792.042362:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.042368:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.042383:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74788864 i_blocks 146096 00000020:00200000:0.0:1789760792.042387:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.042399:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.042443:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74788864, count: 4096 00000080:00200000:0.0:1789760792.042450:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74788864, bytes: 4096 00000080:00200000:0.0:1789760792.042463:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.042472:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.042475:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74788864, 74792960) 0 0 00000080:00200000:0.0:1789760792.042480:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.042497:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18259, 18259] 00020000:00200000:0.0:1789760792.042506:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74788864 chunk: [74788864, 74792960) 74792960, 4096 00020000:00200000:0.0:1789760792.042584:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.042609:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18259, 18259] 00000008:00010000:0.0:1789760792.042627:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.042633:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.042681:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74788864, 74792960) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.042696:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74788864=0x4753000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.042755:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.042757:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74792960 00000020:00010000:0.0:1789760792.042766:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.042769:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.042772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.042776:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.042779:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.042787:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.042794:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.042811:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760792.043110:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40700 - object 0x240000400:5 offset 74788864<>74792960 00000100:00100000:1.0:1789760792.043166:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565568384:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.061299:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985567 gen 3 00000008:00010000:1.0:1789760792.063937:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760792.063955:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.063972:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.064001:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760792.064041:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565568384:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.064937:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18259, 18259] 00000080:00200000:2.0:1789760792.064956:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.064963:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.064981:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74792960 i_blocks 146104 00000020:00200000:2.0:1789760792.064984:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.064997:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.065035:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74792960, count: 4096 00000080:00200000:2.0:1789760792.065042:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74792960, bytes: 4096 00000080:00200000:2.0:1789760792.065056:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.065065:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.065067:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74792960, 74797056) 0 0 00000080:00200000:2.0:1789760792.065071:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.065093:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18260, 18260] 00020000:00200000:2.0:1789760792.065103:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74792960 chunk: [74792960, 74797056) 74797056, 4096 00020000:00200000:2.0:1789760792.065108:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.065130:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18260, 18260] 00000008:00010000:2.0:1789760792.065148:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.065155:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.065169:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74792960, 74797056) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.065183:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74792960=0x4754000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.065247:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.065251:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74797056 00000020:00010000:2.0:1789760792.065259:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.065263:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.065266:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.065269:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.065272:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.065280:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.065286:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.065306:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760792.065870:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74792960<>74797056 00000100:00100000:2.0:1789760792.065990:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760792.066016:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565568512:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.086074:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985568 gen 3 00000008:00010000:2.0:1789760792.086135:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760792.086145:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.086159:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.089056:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760792.089106:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565568512:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760792.089306:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18260, 18260] 00000080:00200000:3.0:1789760792.089322:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760792.089327:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760792.089341:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74797056 i_blocks 146112 00000020:00200000:3.0:1789760792.089344:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760792.089356:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760792.089391:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74797056, count: 4096 00000080:00200000:3.0:1789760792.089399:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74797056, bytes: 4096 00000080:00200000:3.0:1789760792.089424:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760792.089432:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760792.089434:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74797056, 74801152) 0 0 00000080:00200000:3.0:1789760792.089438:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760792.089453:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18261, 18261] 00020000:00200000:3.0:1789760792.089460:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74797056 chunk: [74797056, 74801152) 74801152, 4096 00020000:00200000:3.0:1789760792.089463:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760792.089480:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18261, 18261] 00000008:00010000:3.0:1789760792.089498:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760792.089504:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760792.089516:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74797056, 74801152) append 0 append lockpos 0 00000080:00200000:3.0:1789760792.089531:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74797056=0x4755000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760792.089592:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760792.089595:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74801152 00000020:00010000:3.0:1789760792.089604:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:3.0:1789760792.089607:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.089611:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.089615:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760792.089617:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760792.089627:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760792.089633:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760792.090110:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760792.090258:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 74797056<>74801152 00000100:00100000:3.0:1789760792.090304:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760792.090323:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565568640:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760792.090777:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760792.100079:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985569 gen 3 00000008:00010000:3.0:1789760792.100133:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760792.100142:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760792.100153:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760792.100181:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760792.100221:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565568640:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.100457:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18261, 18261] 00000080:00200000:2.0:1789760792.100475:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.100481:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.100509:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74801152 i_blocks 146120 00000020:00200000:2.0:1789760792.100513:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.100524:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.100562:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74801152, count: 4096 00000080:00200000:2.0:1789760792.100570:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74801152, bytes: 4096 00000080:00200000:2.0:1789760792.100581:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.100589:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.100592:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74801152, 74805248) 0 0 00000080:00200000:2.0:1789760792.100597:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.100616:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18262, 18262] 00020000:00200000:2.0:1789760792.100627:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74801152 chunk: [74801152, 74805248) 74805248, 4096 00020000:00200000:2.0:1789760792.100632:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.100703:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18262, 18262] 00000008:00010000:2.0:1789760792.100722:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.100729:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.100741:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74801152, 74805248) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.100879:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74801152=0x4756000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.100944:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.100947:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74805248 00000020:00010000:2.0:1789760792.100958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.100962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.100965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.100968:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.100971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.100980:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.100987:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.101007:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760792.101131:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74801152<>74805248 00000100:00100000:3.0:1789760792.101214:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565568768:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.112628:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985570 gen 3 00000008:00010000:2.0:1789760792.112754:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760792.112763:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.112774:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.112853:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760792.112888:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565568768:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760792.113032:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18262, 18262] 00000080:00200000:1.0:1789760792.113051:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760792.113058:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.113073:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74805248 i_blocks 146128 00000020:00200000:1.0:1789760792.113077:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.113087:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.113120:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74805248, count: 4096 00000080:00200000:1.0:1789760792.113129:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74805248, bytes: 4096 00000080:00200000:1.0:1789760792.113143:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.113151:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.113153:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74805248, 74809344) 0 0 00000080:00200000:1.0:1789760792.113158:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.113176:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18263, 18263] 00020000:00200000:1.0:1789760792.113185:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74805248 chunk: [74805248, 74809344) 74809344, 4096 00020000:00200000:1.0:1789760792.113189:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.113808:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18263, 18263] 00000008:00010000:1.0:1789760792.113836:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.113843:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.113856:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74805248, 74809344) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.113871:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74805248=0x4757000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.114830:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.114839:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74809344 00000020:00010000:1.0:1789760792.114853:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.114857:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.114862:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.114865:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.114867:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.114879:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.114887:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.114911:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760792.115045:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74805248<>74809344 00000100:00100000:1.0:1789760792.115096:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760792.115122:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565568896:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.128431:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985571 gen 3 00000008:00010000:1.0:1789760792.128581:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760792.128592:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.128602:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.128729:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760792.128771:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565568896:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.128965:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18263, 18263] 00000080:00200000:0.0:1789760792.128981:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.128987:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.129003:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74809344 i_blocks 146136 00000020:00200000:0.0:1789760792.129008:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.129017:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.129054:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74809344, count: 4096 00000080:00200000:0.0:1789760792.129062:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74809344, bytes: 4096 00000080:00200000:0.0:1789760792.129074:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.129084:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.129086:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74809344, 74813440) 0 0 00000080:00200000:0.0:1789760792.129090:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.129109:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18264, 18264] 00020000:00200000:0.0:1789760792.129119:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74809344 chunk: [74809344, 74813440) 74813440, 4096 00020000:00200000:0.0:1789760792.129123:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.129144:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18264, 18264] 00000008:00010000:0.0:1789760792.129162:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.129168:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.129180:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74809344, 74813440) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.129195:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74809344=0x4758000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.129255:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.129259:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74813440 00000020:00010000:0.0:1789760792.129268:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.129271:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.129274:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.129278:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.129281:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.129289:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.129296:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.129314:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760792.129633:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43100 - object 0x240000400:5 offset 74809344<>74813440 00000100:00100000:1.0:1789760792.129739:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565569024:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.152488:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985572 gen 3 00000008:00010000:1.0:1789760792.152547:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760792.152556:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.152569:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.152598:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:1.0:1789760792.152637:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565569024:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.153023:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18264, 18264] 00000080:00200000:0.0:1789760792.153040:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.153046:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.153062:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74813440 i_blocks 146144 00000020:00200000:0.0:1789760792.153065:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.153077:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.153110:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74813440, count: 4096 00000080:00200000:0.0:1789760792.153118:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74813440, bytes: 4096 00000080:00200000:0.0:1789760792.153131:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.153140:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.153143:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74813440, 74817536) 0 0 00000080:00200000:0.0:1789760792.153147:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.153165:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18265, 18265] 00020000:00200000:0.0:1789760792.153175:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74813440 chunk: [74813440, 74817536) 74817536, 4096 00020000:00200000:0.0:1789760792.153179:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.153201:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18265, 18265] 00000008:00010000:0.0:1789760792.153220:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.153227:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.153239:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74813440, 74817536) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.153254:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74813440=0x4759000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.153318:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.153321:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74817536 00000020:00010000:0.0:1789760792.153330:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.153334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.153337:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.153340:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.153889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.153904:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.153915:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.153942:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d300 00000008:00100000:0.0:1789760792.154509:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41f80 - object 0x240000400:5 offset 74813440<>74817536 00000100:00100000:1.0:1789760792.155234:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569152:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.168708:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985573 gen 3 00000008:00010000:1.0:1789760792.172903:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760792.172918:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.172933:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.172951:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760792.172999:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569152:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.174294:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18265, 18265] 00000080:00200000:0.0:1789760792.174318:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.174324:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.174342:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74817536 i_blocks 146152 00000020:00200000:0.0:1789760792.174355:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.174369:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.174400:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74817536, count: 4096 00000080:00200000:0.0:1789760792.174415:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74817536, bytes: 4096 00000080:00200000:0.0:1789760792.174429:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.174438:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.174440:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74817536, 74821632) 0 0 00000080:00200000:0.0:1789760792.174445:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.174461:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18266, 18266] 00020000:00200000:0.0:1789760792.174470:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74817536 chunk: [74817536, 74821632) 74821632, 4096 00020000:00200000:0.0:1789760792.174474:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.174494:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18266, 18266] 00000008:00010000:0.0:1789760792.174512:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.174518:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.174530:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74817536, 74821632) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.174544:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74817536=0x475a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.174606:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.174609:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74821632 00000020:00010000:0.0:1789760792.174617:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.174621:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.174625:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.174628:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.174630:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.174638:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.174687:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.174707:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760792.174841:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 74817536<>74821632 00000100:00100000:1.0:1789760792.176358:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760792.176392:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569280:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.188826:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985574 gen 3 00000008:00010000:1.0:1789760792.188873:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760792.188881:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.188892:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.188912:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:1.0:1789760792.188942:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569280:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.189101:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18266, 18266] 00000080:00200000:0.0:1789760792.189118:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.189124:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.189139:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74821632 i_blocks 146160 00000020:00200000:0.0:1789760792.189142:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.189153:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.189196:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74821632, count: 4096 00000080:00200000:0.0:1789760792.189204:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74821632, bytes: 4096 00000080:00200000:0.0:1789760792.189214:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.189222:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.189224:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74821632, 74825728) 0 0 00000080:00200000:0.0:1789760792.189229:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.189247:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18267, 18267] 00020000:00200000:0.0:1789760792.189256:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74821632 chunk: [74821632, 74825728) 74825728, 4096 00020000:00200000:0.0:1789760792.189261:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.189280:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18267, 18267] 00000008:00010000:0.0:1789760792.189299:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.189305:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.189316:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74821632, 74825728) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.189329:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74821632=0x475b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.189401:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.189413:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74825728 00000020:00010000:0.0:1789760792.189422:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.189425:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.189429:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.189433:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.189436:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.189443:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.189450:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.189467:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760792.189593:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 74821632<>74825728 00000100:00100000:1.0:1789760792.192117:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569408:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.205595:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985575 gen 3 00000008:00010000:1.0:1789760792.205660:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760792.205668:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.205679:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.205692:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760792.205723:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569408:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.206024:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18267, 18267] 00000080:00200000:0.0:1789760792.206041:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.206047:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.206061:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74825728 i_blocks 146168 00000020:00200000:0.0:1789760792.206065:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.206076:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.206109:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74825728, count: 4096 00000080:00200000:0.0:1789760792.206116:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74825728, bytes: 4096 00000080:00200000:0.0:1789760792.206129:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.206138:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.206140:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74825728, 74829824) 0 0 00000080:00200000:0.0:1789760792.206145:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.206171:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18268, 18268] 00020000:00200000:0.0:1789760792.206181:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74825728 chunk: [74825728, 74829824) 74829824, 4096 00020000:00200000:0.0:1789760792.206186:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.206206:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18268, 18268] 00000008:00010000:0.0:1789760792.206226:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.206231:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.206245:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74825728, 74829824) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.206259:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74825728=0x475c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.206323:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.206326:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74829824 00000020:00010000:0.0:1789760792.206333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.206337:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.206340:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.206343:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.206346:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.206352:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.206358:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.206374:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760792.206788:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 74825728<>74829824 00000100:00100000:1.0:1789760792.206893:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760792.206915:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569536:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.218062:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985576 gen 3 00000008:00010000:1.0:1789760792.218111:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760792.218119:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.218131:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.218156:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760792.218188:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569536:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.218301:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18268, 18268] 00000080:00200000:0.0:1789760792.218319:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.218324:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.218339:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74829824 i_blocks 146176 00000020:00200000:0.0:1789760792.218353:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.218365:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.218402:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74829824, count: 4096 00000080:00200000:0.0:1789760792.218426:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74829824, bytes: 4096 00000080:00200000:0.0:1789760792.218437:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.218445:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.218448:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74829824, 74833920) 0 0 00000080:00200000:0.0:1789760792.218452:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.218470:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18269, 18269] 00020000:00200000:0.0:1789760792.218479:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74829824 chunk: [74829824, 74833920) 74833920, 4096 00020000:00200000:0.0:1789760792.218492:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.218513:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18269, 18269] 00000008:00010000:0.0:1789760792.218532:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.218538:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.218550:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74829824, 74833920) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.218565:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74829824=0x475d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.218621:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.218624:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74833920 00000020:00010000:0.0:1789760792.218632:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.218635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.218638:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.218888:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.218892:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.218902:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.218910:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.218931:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760792.219058:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 74829824<>74833920 00000100:00100000:1.0:1789760792.219119:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569664:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.227291:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985577 gen 3 00000008:00010000:1.0:1789760792.227337:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760792.227354:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.227365:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.227382:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000080:00200000:0.0:1789760792.228330:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18269, 18269] 00000080:00200000:0.0:1789760792.228360:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.228367:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.228385:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74833920 i_blocks 146184 00000020:00200000:0.0:1789760792.228389:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.228401:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000100:00100000:1.0:1789760792.228769:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569664:192.168.203.147@tcp:4:dd.0 00000080:00200008:0.0:1789760792.229022:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74833920, count: 4096 00000080:00200000:0.0:1789760792.229033:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74833920, bytes: 4096 00000080:00200000:0.0:1789760792.229048:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.229056:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.229059:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74833920, 74838016) 0 0 00000080:00200000:0.0:1789760792.229064:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.229080:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18270, 18270] 00020000:00200000:0.0:1789760792.229089:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74833920 chunk: [74833920, 74838016) 74838016, 4096 00020000:00200000:0.0:1789760792.229094:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.229114:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18270, 18270] 00000008:00010000:0.0:1789760792.229130:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.229135:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.229146:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74833920, 74838016) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.229159:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74833920=0x475e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.229221:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.229224:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74838016 00000020:00010000:0.0:1789760792.229232:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.229236:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.229238:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.229241:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.229245:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.229252:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.229259:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.229277:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760792.229901:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 74833920<>74838016 00000100:00100000:1.0:1789760792.230286:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760792.230596:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569792:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.245714:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985578 gen 3 00000008:00010000:1.0:1789760792.245760:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760792.245768:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.245779:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.248546:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760792.248608:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565569792:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.249549:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18270, 18270] 00000080:00200000:2.0:1789760792.249570:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.249576:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.249594:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74838016 i_blocks 146192 00000020:00200000:2.0:1789760792.249599:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.249614:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.249666:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74838016, count: 4096 00000080:00200000:2.0:1789760792.249674:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74838016, bytes: 4096 00000080:00200000:2.0:1789760792.249991:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.250001:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.250004:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74838016, 74842112) 0 0 00000080:00200000:2.0:1789760792.250009:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.250026:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18271, 18271] 00020000:00200000:2.0:1789760792.250036:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74838016 chunk: [74838016, 74842112) 74842112, 4096 00020000:00200000:2.0:1789760792.250040:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.250063:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18271, 18271] 00000008:00010000:2.0:1789760792.250081:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.250087:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.250100:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74838016, 74842112) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.250113:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74838016=0x475f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.250175:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.250178:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74842112 00000020:00010000:2.0:1789760792.250186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.250190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.250193:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.250196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.250199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.250207:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.250213:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.250241:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760792.250365:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74838016<>74842112 00000100:00100000:3.0:1789760792.250542:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565569920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760792.267534:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985579 gen 3 00000008:00010000:3.0:1789760792.267587:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:3.0:1789760792.267597:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760792.267608:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760792.267636:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:3.0:1789760792.267757:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565569920:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760792.268267:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18271, 18271] 00000080:00200000:1.0:1789760792.268287:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760792.268293:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.268309:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74842112 i_blocks 146200 00000020:00200000:1.0:1789760792.268312:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.268323:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.268357:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74842112, count: 4096 00000080:00200000:1.0:1789760792.268363:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74842112, bytes: 4096 00000080:00200000:1.0:1789760792.268374:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.268382:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.268384:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74842112, 74846208) 0 0 00000080:00200000:1.0:1789760792.268389:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.268485:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18272, 18272] 00020000:00200000:1.0:1789760792.268551:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74842112 chunk: [74842112, 74846208) 74846208, 4096 00020000:00200000:1.0:1789760792.268556:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.268577:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18272, 18272] 00000008:00010000:1.0:1789760792.268595:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.268601:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.268613:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74842112, 74846208) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.268626:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74842112=0x4760000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.268713:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.268716:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74846208 00000020:00010000:1.0:1789760792.268724:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.268727:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.268730:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.268734:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.268737:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.268745:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.268753:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.268771:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760792.268952:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74842112<>74846208 00000100:00100000:0.0:1789760792.269035:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565570048:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.286601:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985580 gen 3 00000008:00010000:1.0:1789760792.287031:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760792.287041:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.287053:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.287086:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760792.287125:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565570048:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760792.287378:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18272, 18272] 00000080:00200000:3.0:1789760792.287396:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760792.287402:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760792.287429:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74846208 i_blocks 146208 00000020:00200000:3.0:1789760792.287433:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760792.287443:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760792.287479:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74846208, count: 4096 00000080:00200000:3.0:1789760792.287487:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74846208, bytes: 4096 00000080:00200000:3.0:1789760792.287500:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760792.287508:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760792.287510:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74846208, 74850304) 0 0 00000080:00200000:3.0:1789760792.287515:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760792.287536:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18273, 18273] 00020000:00200000:3.0:1789760792.287546:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74846208 chunk: [74846208, 74850304) 74850304, 4096 00020000:00200000:3.0:1789760792.287551:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760792.287573:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18273, 18273] 00000008:00010000:3.0:1789760792.287591:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760792.287598:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760792.287611:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74846208, 74850304) append 0 append lockpos 0 00000080:00200000:3.0:1789760792.287626:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74846208=0x4761000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760792.287905:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760792.287909:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74850304 00000020:00010000:3.0:1789760792.287976:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:3.0:1789760792.287981:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.287985:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.287989:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760792.287992:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760792.288001:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760792.288008:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760792.288026:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760792.288147:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 74846208<>74850304 00000100:00100000:3.0:1789760792.288198:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760792.288287:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565570176:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760792.303308:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985581 gen 3 00000008:00010000:3.0:1789760792.303363:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760792.303372:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760792.303384:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760792.303424:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760792.303455:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565570176:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760792.303953:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18273, 18273] 00000080:00200000:1.0:1789760792.303972:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760792.303979:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.303995:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74850304 i_blocks 146216 00000020:00200000:1.0:1789760792.304000:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.304012:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.304047:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74850304, count: 4096 00000080:00200000:1.0:1789760792.304055:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74850304, bytes: 4096 00000080:00200000:1.0:1789760792.304066:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.304074:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.304077:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74850304, 74854400) 0 0 00000080:00200000:1.0:1789760792.304080:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.304096:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18274, 18274] 00020000:00200000:1.0:1789760792.304104:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74850304 chunk: [74850304, 74854400) 74854400, 4096 00020000:00200000:1.0:1789760792.304108:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.304130:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18274, 18274] 00000008:00010000:1.0:1789760792.304147:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.304154:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.304166:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74850304, 74854400) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.304179:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74850304=0x4762000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.304240:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.304243:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74854400 00000020:00010000:1.0:1789760792.304251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.304255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.304258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.304261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.304264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.304271:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.304278:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.304297:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760792.304494:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74850304<>74854400 00000100:00100000:0.0:1789760792.305254:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565570304:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.322965:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985582 gen 3 00000008:00010000:1.0:1789760792.323097:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760792.323106:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.323117:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.323211:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760792.323241:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565570304:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.323326:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18274, 18274] 00000080:00200000:2.0:1789760792.323342:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.323348:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.323363:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74854400 i_blocks 146224 00000020:00200000:2.0:1789760792.323367:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.323378:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.323952:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74854400, count: 4096 00000080:00200000:2.0:1789760792.323964:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74854400, bytes: 4096 00000080:00200000:2.0:1789760792.323979:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.323988:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.323990:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74854400, 74858496) 0 0 00000080:00200000:2.0:1789760792.323995:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.324011:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18275, 18275] 00020000:00200000:2.0:1789760792.324020:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74854400 chunk: [74854400, 74858496) 74858496, 4096 00020000:00200000:2.0:1789760792.324024:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.324048:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18275, 18275] 00000008:00010000:2.0:1789760792.324067:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.324073:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.324085:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74854400, 74858496) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.324099:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74854400=0x4763000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.324158:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.324161:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74858496 00000020:00010000:2.0:1789760792.324169:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.324173:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.324176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.324180:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.324183:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.324192:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.324200:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.324222:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760792.324344:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 74854400<>74858496 00000100:00100000:3.0:1789760792.324583:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565570432:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.340479:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985583 gen 3 00000008:00010000:2.0:1789760792.344632:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760792.344642:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.344668:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.344799:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760792.344838:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565570432:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760792.345053:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18275, 18275] 00000080:00200000:3.0:1789760792.345070:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760792.345076:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760792.345093:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74858496 i_blocks 146232 00000020:00200000:3.0:1789760792.345096:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760792.345108:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760792.345145:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74858496, count: 4096 00000080:00200000:3.0:1789760792.345153:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74858496, bytes: 4096 00000080:00200000:3.0:1789760792.345179:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760792.345188:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760792.345191:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74858496, 74862592) 0 0 00000080:00200000:3.0:1789760792.345195:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760792.345213:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18276, 18276] 00020000:00200000:3.0:1789760792.345223:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74858496 chunk: [74858496, 74862592) 74862592, 4096 00020000:00200000:3.0:1789760792.345228:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760792.345247:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18276, 18276] 00000008:00010000:3.0:1789760792.345266:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760792.345273:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760792.345285:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74858496, 74862592) append 0 append lockpos 0 00000080:00200000:3.0:1789760792.345301:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74858496=0x4764000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760792.345375:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760792.345378:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74862592 00000020:00010000:3.0:1789760792.345385:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:3.0:1789760792.345389:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.345396:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.345400:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760792.345402:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760792.345421:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760792.345428:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760792.345446:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760792.345572:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 74858496<>74862592 00000100:00100000:2.0:1789760792.345695:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565570560:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.367487:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985584 gen 3 00000008:00010000:2.0:1789760792.367549:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760792.367557:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.367569:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.367592:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760792.367621:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565570560:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760792.367883:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18276, 18276] 00000080:00200000:3.0:1789760792.367901:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760792.367906:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760792.367922:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74862592 i_blocks 146240 00000020:00200000:3.0:1789760792.367925:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760792.367936:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760792.367970:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74862592, count: 4096 00000080:00200000:3.0:1789760792.367979:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74862592, bytes: 4096 00000080:00200000:3.0:1789760792.367990:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760792.367998:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760792.368001:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74862592, 74866688) 0 0 00000080:00200000:3.0:1789760792.368005:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760792.368021:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18277, 18277] 00020000:00200000:3.0:1789760792.368032:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74862592 chunk: [74862592, 74866688) 74866688, 4096 00020000:00200000:3.0:1789760792.368037:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760792.368057:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18277, 18277] 00000008:00010000:3.0:1789760792.368076:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760792.368082:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760792.368093:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74862592, 74866688) append 0 append lockpos 0 00000080:00200000:3.0:1789760792.368108:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74862592=0x4765000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760792.368168:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760792.368171:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74866688 00000020:00010000:3.0:1789760792.368178:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:3.0:1789760792.368182:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.368185:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.368188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760792.368191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760792.368199:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760792.368205:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760792.368223:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760792.368353:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 74862592<>74866688 00000100:00100000:2.0:1789760792.368686:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565570688:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.382888:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985585 gen 3 00000008:00010000:2.0:1789760792.382943:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760792.382950:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.382962:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.383030:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760792.383061:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565570688:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760792.383144:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18277, 18277] 00000080:00200000:3.0:1789760792.383161:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760792.383168:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760792.383183:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74866688 i_blocks 146248 00000020:00200000:3.0:1789760792.383186:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760792.383196:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760792.383229:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74866688, count: 4096 00000080:00200000:3.0:1789760792.383237:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74866688, bytes: 4096 00000080:00200000:3.0:1789760792.383246:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760792.383254:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760792.383257:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74866688, 74870784) 0 0 00000080:00200000:3.0:1789760792.383261:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760792.383278:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18278, 18278] 00020000:00200000:3.0:1789760792.383288:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74866688 chunk: [74866688, 74870784) 74870784, 4096 00020000:00200000:3.0:1789760792.383292:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760792.383311:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18278, 18278] 00000008:00010000:3.0:1789760792.383328:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760792.383333:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760792.383833:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74866688, 74870784) append 0 append lockpos 0 00000080:00200000:3.0:1789760792.383852:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74866688=0x4766000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760792.384066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760792.384069:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74870784 00000020:00010000:3.0:1789760792.384078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:3.0:1789760792.384082:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.384085:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.384088:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760792.384091:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760792.384100:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760792.384108:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760792.384126:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760792.384257:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66d80 - object 0x240000400:5 offset 74866688<>74870784 00000100:00100000:2.0:1789760792.384325:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565570816:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.400835:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985586 gen 3 00000008:00010000:2.0:1789760792.400890:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760792.400899:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.400911:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.400936:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760792.401035:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565570816:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760792.401104:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18278, 18278] 00000080:00200000:3.0:1789760792.401121:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760792.401126:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760792.401138:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74870784 i_blocks 146256 00000020:00200000:3.0:1789760792.401141:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760792.401150:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760792.401179:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74870784, count: 4096 00000080:00200000:3.0:1789760792.401185:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74870784, bytes: 4096 00000080:00200000:3.0:1789760792.401193:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760792.401199:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760792.401201:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74870784, 74874880) 0 0 00000080:00200000:3.0:1789760792.401205:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760792.401218:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18279, 18279] 00020000:00200000:3.0:1789760792.401225:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74870784 chunk: [74870784, 74874880) 74874880, 4096 00020000:00200000:3.0:1789760792.401229:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760792.401248:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18279, 18279] 00000008:00010000:3.0:1789760792.401265:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760792.401271:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760792.401283:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74870784, 74874880) append 0 append lockpos 0 00000080:00200000:3.0:1789760792.401298:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74870784=0x4767000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760792.402220:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760792.402229:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74874880 00000020:00010000:3.0:1789760792.402243:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:3.0:1789760792.402247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.402251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.402256:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760792.402259:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760792.402272:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760792.402281:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760792.402310:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760792.402714:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 74870784<>74874880 00000100:00100000:2.0:1789760792.403087:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565570944:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.419166:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985587 gen 3 00000008:00010000:2.0:1789760792.419217:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760792.419236:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.419250:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.419267:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760792.419297:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565570944:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760792.419684:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18279, 18279] 00000080:00200000:3.0:1789760792.419701:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760792.419708:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760792.419722:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74874880 i_blocks 146264 00000020:00200000:3.0:1789760792.419726:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760792.419735:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760792.419780:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74874880, count: 4096 00000080:00200000:3.0:1789760792.419788:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74874880, bytes: 4096 00000080:00200000:3.0:1789760792.419799:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760792.419807:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760792.419810:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74874880, 74878976) 0 0 00000080:00200000:3.0:1789760792.419815:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760792.419833:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18280, 18280] 00020000:00200000:3.0:1789760792.419843:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74874880 chunk: [74874880, 74878976) 74878976, 4096 00020000:00200000:3.0:1789760792.419848:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760792.419867:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18280, 18280] 00000008:00010000:3.0:1789760792.419883:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760792.419890:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760792.419902:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74874880, 74878976) append 0 append lockpos 0 00000080:00200000:3.0:1789760792.419916:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74874880=0x4768000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760792.419974:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760792.419977:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74878976 00000020:00010000:3.0:1789760792.419986:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:3.0:1789760792.419990:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.419992:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760792.419996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760792.419998:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760792.420007:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760792.420014:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760792.420031:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760792.420151:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 74874880<>74878976 00000100:00100000:2.0:1789760792.420592:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760792.420615:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565571072:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.437157:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985588 gen 3 00000008:00010000:2.0:1789760792.437209:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760792.437219:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.437231:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.437256:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760792.437289:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565571072:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760792.437473:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18280, 18280] 00000080:00200000:3.0:1789760792.437500:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760792.437506:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760792.437522:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74878976 i_blocks 146272 00000020:00200000:3.0:1789760792.437526:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760792.437536:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760792.437567:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74878976, count: 4096 00000080:00200000:3.0:1789760792.437574:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74878976, bytes: 4096 00000080:00200000:3.0:1789760792.437584:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760792.437592:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760792.437595:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74878976, 74883072) 0 0 00000080:00200000:3.0:1789760792.437599:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760792.437614:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18281, 18281] 00020000:00200000:3.0:1789760792.437624:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74878976 chunk: [74878976, 74883072) 74883072, 4096 00020000:00200000:3.0:1789760792.437629:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.437932:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18281, 18281] 00000008:00010000:2.0:1789760792.437953:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.437958:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.437970:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74878976, 74883072) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.437984:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74878976=0x4769000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.438036:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.438039:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74883072 00000020:00010000:2.0:1789760792.438047:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.438051:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.438054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.438058:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.438061:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.438070:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.438076:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.438089:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760792.438212:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 74878976<>74883072 00000100:00100000:3.0:1789760792.438306:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760792.438328:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565571200:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760792.457895:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985589 gen 3 00000008:00010000:2.0:1789760792.459223:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760792.459238:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.459250:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.459356:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760792.459392:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565571200:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.459606:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18281, 18281] 00000080:00200000:0.0:1789760792.459621:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.459627:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.459730:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74883072 i_blocks 146280 00000020:00200000:0.0:1789760792.459735:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.459745:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.459780:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74883072, count: 4096 00000080:00200000:0.0:1789760792.459788:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74883072, bytes: 4096 00000080:00200000:0.0:1789760792.459803:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.459812:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.459814:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74883072, 74887168) 0 0 00000080:00200000:0.0:1789760792.459818:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.459836:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18282, 18282] 00020000:00200000:0.0:1789760792.459846:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74883072 chunk: [74883072, 74887168) 74887168, 4096 00020000:00200000:0.0:1789760792.459850:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.459871:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18282, 18282] 00000008:00010000:0.0:1789760792.459890:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.459896:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.459908:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74883072, 74887168) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.459921:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74883072=0x476a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.459983:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.459986:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74887168 00000020:00010000:0.0:1789760792.459994:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.459997:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.460000:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.460003:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.460007:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.460310:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.460319:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.460343:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760792.460480:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 74883072<>74887168 00000100:00100000:1.0:1789760792.460674:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565571328:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.474384:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985590 gen 3 00000008:00010000:1.0:1789760792.474446:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760792.474456:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.474468:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.474489:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:1.0:1789760792.474522:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565571328:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.475116:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18282, 18282] 00000080:00200000:0.0:1789760792.475135:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.475141:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.475158:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74887168 i_blocks 146288 00000020:00200000:0.0:1789760792.475161:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.475172:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.475206:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74887168, count: 4096 00000080:00200000:0.0:1789760792.475213:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74887168, bytes: 4096 00000080:00200000:0.0:1789760792.475225:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.475233:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.475236:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74887168, 74891264) 0 0 00000080:00200000:0.0:1789760792.475240:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.475257:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18283, 18283] 00020000:00200000:0.0:1789760792.475266:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74887168 chunk: [74887168, 74891264) 74891264, 4096 00020000:00200000:0.0:1789760792.475271:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.475289:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18283, 18283] 00000008:00010000:0.0:1789760792.475308:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.475313:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.475325:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74887168, 74891264) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.475350:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74887168=0x476b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.475421:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.475424:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74891264 00000020:00010000:0.0:1789760792.475432:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.475436:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.475440:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.475443:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.475447:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.475454:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.475460:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.475478:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:0.0:1789760792.475599:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 74887168<>74891264 00000100:00100000:1.0:1789760792.475807:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565571456:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.496527:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985591 gen 3 00000008:00010000:1.0:1789760792.496678:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760792.496687:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.496698:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.496725:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760792.496766:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565571456:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.497058:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18283, 18283] 00000080:00200000:2.0:1789760792.497075:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.497081:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.497096:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74891264 i_blocks 146296 00000020:00200000:2.0:1789760792.497099:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.497111:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.497146:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74891264, count: 4096 00000080:00200000:2.0:1789760792.497152:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74891264, bytes: 4096 00000080:00200000:2.0:1789760792.497162:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.497170:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.497172:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74891264, 74895360) 0 0 00000080:00200000:2.0:1789760792.497177:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.497194:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18284, 18284] 00020000:00200000:2.0:1789760792.497203:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74891264 chunk: [74891264, 74895360) 74895360, 4096 00020000:00200000:2.0:1789760792.497207:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.497226:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18284, 18284] 00000008:00010000:2.0:1789760792.497240:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.497246:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.497257:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74891264, 74895360) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.497270:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74891264=0x476c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.497325:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.497328:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74895360 00000020:00010000:2.0:1789760792.497335:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.497338:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.497341:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.497344:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.497347:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.497354:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.497360:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.497375:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760792.497507:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 74891264<>74895360 00000100:00100000:2.0:1789760792.497587:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565571584:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.509510:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985592 gen 3 00000008:00010000:2.0:1789760792.509562:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760792.509570:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.509581:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.509609:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760792.509642:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565571584:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.509811:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18284, 18284] 00000080:00200000:0.0:1789760792.509828:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.509833:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.509848:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74895360 i_blocks 146304 00000020:00200000:0.0:1789760792.509851:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.509861:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.510020:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74895360, count: 4096 00000080:00200000:0.0:1789760792.510030:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74895360, bytes: 4096 00000080:00200000:0.0:1789760792.510042:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.510051:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.510054:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74895360, 74899456) 0 0 00000080:00200000:0.0:1789760792.510059:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.510076:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18285, 18285] 00020000:00200000:0.0:1789760792.510085:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74895360 chunk: [74895360, 74899456) 74899456, 4096 00020000:00200000:0.0:1789760792.510090:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.510111:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18285, 18285] 00000008:00010000:0.0:1789760792.510130:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.510135:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.510148:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74895360, 74899456) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.510163:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74895360=0x476d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.510223:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.510227:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74899456 00000020:00010000:0.0:1789760792.510235:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.510239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.510242:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.510244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.510248:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.510255:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.510262:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.510278:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:0.0:1789760792.511707:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 74895360<>74899456 00000100:00100000:1.0:1789760792.514706:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760792.514746:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565571712:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.535617:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985593 gen 3 00000008:00010000:1.0:1789760792.535713:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760792.535721:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.535732:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.535745:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760792.535775:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565571712:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.535901:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18285, 18285] 00000080:00200000:0.0:1789760792.535918:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.535924:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.535940:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74899456 i_blocks 146312 00000020:00200000:0.0:1789760792.535943:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.535955:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.535987:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74899456, count: 4096 00000080:00200000:0.0:1789760792.535994:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74899456, bytes: 4096 00000080:00200000:0.0:1789760792.536008:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.536017:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.536019:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74899456, 74903552) 0 0 00000080:00200000:0.0:1789760792.536023:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.536041:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18286, 18286] 00020000:00200000:0.0:1789760792.536051:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74899456 chunk: [74899456, 74903552) 74903552, 4096 00020000:00200000:0.0:1789760792.536055:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.536076:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18286, 18286] 00000008:00010000:0.0:1789760792.536096:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.536102:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.536114:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74899456, 74903552) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.536128:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74899456=0x476e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.536185:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.536188:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74903552 00000020:00010000:0.0:1789760792.536196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.536199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.536202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.536206:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.536209:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.536217:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.536239:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.536255:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760792.536602:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41f80 - object 0x240000400:5 offset 74899456<>74903552 00000100:00100000:1.0:1789760792.536682:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565571840:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.553168:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985594 gen 3 00000008:00010000:1.0:1789760792.553245:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760792.553254:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.553266:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.553280:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:1.0:1789760792.553316:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565571840:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.553669:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18286, 18286] 00000080:00200000:0.0:1789760792.553687:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.553693:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.553710:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74903552 i_blocks 146320 00000020:00200000:0.0:1789760792.553714:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.553725:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.553757:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74903552, count: 4096 00000080:00200000:0.0:1789760792.553765:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74903552, bytes: 4096 00000080:00200000:0.0:1789760792.553780:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.553789:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.553791:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74903552, 74907648) 0 0 00000080:00200000:0.0:1789760792.553796:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.553815:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18287, 18287] 00020000:00200000:0.0:1789760792.553824:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74903552 chunk: [74903552, 74907648) 74907648, 4096 00020000:00200000:0.0:1789760792.553829:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.553850:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18287, 18287] 00000008:00010000:0.0:1789760792.553869:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.553886:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.554130:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74903552, 74907648) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.554146:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74903552=0x476f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.554209:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.554212:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74907648 00000020:00010000:0.0:1789760792.554231:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.554236:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.554239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.554242:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.554246:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.554255:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.554262:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.554284:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760792.554422:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43100 - object 0x240000400:5 offset 74903552<>74907648 00000100:00100000:1.0:1789760792.554629:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565571968:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.570094:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985595 gen 3 00000008:00010000:1.0:1789760792.570150:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760792.570158:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.570170:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.570256:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760792.570289:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565571968:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.573286:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18287, 18287] 00000080:00200000:0.0:1789760792.573316:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.573324:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.573343:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74907648 i_blocks 146328 00000020:00200000:0.0:1789760792.573349:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.573369:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.573415:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74907648, count: 4096 00000080:00200000:0.0:1789760792.573424:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74907648, bytes: 4096 00000080:00200000:0.0:1789760792.573439:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.573447:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.573450:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74907648, 74911744) 0 0 00000080:00200000:0.0:1789760792.573454:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.573472:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18288, 18288] 00020000:00200000:0.0:1789760792.573482:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74907648 chunk: [74907648, 74911744) 74911744, 4096 00020000:00200000:0.0:1789760792.573486:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.573507:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18288, 18288] 00000008:00010000:0.0:1789760792.573528:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.573535:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.573547:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74907648, 74911744) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.573561:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74907648=0x4770000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.573637:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.573640:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74911744 00000020:00010000:0.0:1789760792.575368:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.575380:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.575384:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.575389:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.575394:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.575419:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.575432:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.575471:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760792.575624:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40700 - object 0x240000400:5 offset 74907648<>74911744 00000100:00100000:1.0:1789760792.575808:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760792.575836:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565572096:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.599769:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985596 gen 3 00000008:00010000:1.0:1789760792.599818:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760792.599826:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.599836:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.599855:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:1.0:1789760792.600008:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565572096:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.600130:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18288, 18288] 00000080:00200000:0.0:1789760792.600148:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.600153:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.600167:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74911744 i_blocks 146336 00000020:00200000:0.0:1789760792.600171:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.600182:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.600216:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74911744, count: 4096 00000080:00200000:0.0:1789760792.600224:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74911744, bytes: 4096 00000080:00200000:0.0:1789760792.600233:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.600242:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.600244:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74911744, 74915840) 0 0 00000080:00200000:0.0:1789760792.600248:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.600266:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18289, 18289] 00020000:00200000:0.0:1789760792.600275:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74911744 chunk: [74911744, 74915840) 74915840, 4096 00020000:00200000:0.0:1789760792.600280:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.600313:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18289, 18289] 00000008:00010000:0.0:1789760792.600332:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.600339:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.600351:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74911744, 74915840) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.600365:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74911744=0x4771000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.600436:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.600439:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74915840 00000020:00010000:0.0:1789760792.600446:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.600450:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.600453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.600456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.600460:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.600468:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.600475:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.600492:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760792.600617:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 74911744<>74915840 00000100:00100000:1.0:1789760792.600687:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565572224:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.618641:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985597 gen 3 00000008:00010000:1.0:1789760792.618779:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760792.618787:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.618798:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.618860:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760792.618902:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565572224:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.624693:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18289, 18289] 00000080:00200000:0.0:1789760792.624722:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.624729:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.624749:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74915840 i_blocks 146344 00000020:00200000:0.0:1789760792.624754:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.624771:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.624805:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74915840, count: 4096 00000080:00200000:0.0:1789760792.624814:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74915840, bytes: 4096 00000080:00200000:0.0:1789760792.624829:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.624838:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.624840:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74915840, 74919936) 0 0 00000080:00200000:0.0:1789760792.624845:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.624863:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18290, 18290] 00020000:00200000:0.0:1789760792.624872:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74915840 chunk: [74915840, 74919936) 74919936, 4096 00020000:00200000:0.0:1789760792.625038:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.625060:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18290, 18290] 00000008:00010000:0.0:1789760792.625078:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.625084:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.625096:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74915840, 74919936) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.625110:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74915840=0x4772000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.625180:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.625183:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74919936 00000020:00010000:0.0:1789760792.625190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.625194:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.625197:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.625200:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.625204:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.625212:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.625220:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.625250:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760792.625725:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42680 - object 0x240000400:5 offset 74915840<>74919936 00000100:00100000:1.0:1789760792.625872:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565572352:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.643566:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985598 gen 3 00000008:00010000:1.0:1789760792.643618:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760792.643627:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.643638:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.643744:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760792.643774:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565572352:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.647378:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18290, 18290] 00000080:00200000:0.0:1789760792.647415:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.647422:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.651217:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74919936 i_blocks 146352 00000020:00200000:1.0:1789760792.651234:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.651257:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.651295:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74919936, count: 4096 00000080:00200000:1.0:1789760792.651306:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74919936, bytes: 4096 00000080:00200000:1.0:1789760792.651322:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.651331:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.651334:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74919936, 74924032) 0 0 00000080:00200000:1.0:1789760792.651339:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.651357:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18291, 18291] 00020000:00200000:1.0:1789760792.651367:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74919936 chunk: [74919936, 74924032) 74924032, 4096 00020000:00200000:1.0:1789760792.651372:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.651395:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18291, 18291] 00000008:00010000:1.0:1789760792.651425:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.651432:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.651444:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74919936, 74924032) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.651459:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74919936=0x4773000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.652057:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.652062:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74924032 00000020:00010000:1.0:1789760792.652073:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.652077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.652080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.652083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.652087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.652095:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.652102:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.652129:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760792.652254:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74919936<>74924032 00000100:00100000:0.0:1789760792.652356:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565572480:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.686187:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985599 gen 3 00000008:00010000:1.0:1789760792.686239:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760792.686248:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.686258:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.686319:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760792.686357:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565572480:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.690020:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18291, 18291] 00000080:00200000:2.0:1789760792.690049:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.690056:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.690075:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74924032 i_blocks 146360 00000020:00200000:2.0:1789760792.690081:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.690098:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.690134:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74924032, count: 4096 00000080:00200000:2.0:1789760792.690144:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74924032, bytes: 4096 00000080:00200000:2.0:1789760792.690160:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.690171:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.690173:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74924032, 74928128) 0 0 00000080:00200000:2.0:1789760792.690178:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.690198:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18292, 18292] 00020000:00200000:2.0:1789760792.690207:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74924032 chunk: [74924032, 74928128) 74928128, 4096 00020000:00200000:2.0:1789760792.690212:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.690238:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18292, 18292] 00000008:00010000:2.0:1789760792.690258:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.690264:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.690276:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74924032, 74928128) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.690292:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74924032=0x4774000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.692146:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.692156:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74928128 00000020:00010000:2.0:1789760792.692172:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.692177:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.692182:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.692186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.692188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.692201:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.692211:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.692244:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760792.692392:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 74924032<>74928128 00000100:00100000:2.0:1789760792.692462:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760792.692491:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565572608:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760792.708364:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985600 gen 3 00000008:00010000:3.0:1789760792.708427:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760792.708436:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760792.708446:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760792.708476:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760792.708692:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565572608:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760792.709664:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18292, 18292] 00000080:00200000:1.0:1789760792.709685:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760792.709692:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.709709:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74928128 i_blocks 146368 00000020:00200000:1.0:1789760792.709714:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.709726:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.709762:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74928128, count: 4096 00000080:00200000:1.0:1789760792.709782:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74928128, bytes: 4096 00000080:00200000:1.0:1789760792.709796:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.709805:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.709807:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74928128, 74932224) 0 0 00000080:00200000:1.0:1789760792.709811:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.709829:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18293, 18293] 00020000:00200000:1.0:1789760792.709839:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74928128 chunk: [74928128, 74932224) 74932224, 4096 00020000:00200000:1.0:1789760792.709844:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.709866:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18293, 18293] 00000008:00010000:1.0:1789760792.709885:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.709891:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.709905:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74928128, 74932224) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.709920:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74928128=0x4775000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.710641:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.710694:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74932224 00000020:00010000:1.0:1789760792.710706:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.710710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.710713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.710717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e6c8 (ffff93f84354e758/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.710720:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e758/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.710731:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.710739:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.711184:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760792.711305:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74928128<>74932224 00000100:00100000:0.0:1789760792.711365:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760792.711606:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565572736:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.728594:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985601 gen 3 00000008:00010000:1.0:1789760792.728688:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:1.0:1789760792.728696:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.728708:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.728719:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:1.0:1789760792.728757:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565572736:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.729384:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18293, 18293] 00000080:00200000:0.0:1789760792.729548:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.729556:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.729575:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74932224 i_blocks 146376 00000020:00200000:0.0:1789760792.729579:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.729590:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.729733:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74932224, count: 4096 00000080:00200000:0.0:1789760792.729743:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74932224, bytes: 4096 00000080:00200000:0.0:1789760792.729871:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.729881:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.729884:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74932224, 74936320) 0 0 00000080:00200000:0.0:1789760792.729889:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.729906:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18294, 18294] 00020000:00200000:0.0:1789760792.729916:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74932224 chunk: [74932224, 74936320) 74936320, 4096 00020000:00200000:0.0:1789760792.729921:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.729944:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18294, 18294] 00000008:00010000:0.0:1789760792.730075:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.730082:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.730095:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74932224, 74936320) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.730110:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74932224=0x4776000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.730175:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.730178:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74936320 00000020:00010000:0.0:1789760792.730186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.730190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.730192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.730196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.730198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.730207:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.730213:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.730235:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760792.730370:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 74932224<>74936320 00000100:00100000:1.0:1789760792.730456:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565572864:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.751475:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985602 gen 3 00000008:00010000:1.0:1789760792.751529:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760792.751538:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.751549:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.751607:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:1.0:1789760792.751639:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565572864:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.755031:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18294, 18294] 00000080:00200000:0.0:1789760792.755058:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.755065:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760792.755102:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74936320 i_blocks 146384 00000020:00200000:0.0:1789760792.755107:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760792.755121:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760792.755152:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74936320, count: 4096 00000080:00200000:0.0:1789760792.755160:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74936320, bytes: 4096 00000080:00200000:0.0:1789760792.755173:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760792.755375:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760792.755379:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74936320, 74940416) 0 0 00000080:00200000:0.0:1789760792.755384:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760792.755485:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18295, 18295] 00020000:00200000:0.0:1789760792.755499:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74936320 chunk: [74936320, 74940416) 74940416, 4096 00020000:00200000:0.0:1789760792.755512:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760792.755535:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18295, 18295] 00000008:00010000:0.0:1789760792.755553:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760792.755559:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760792.755572:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74936320, 74940416) append 0 append lockpos 0 00000080:00200000:0.0:1789760792.755588:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74936320=0x4777000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760792.755683:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760792.755686:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74940416 00000020:00010000:0.0:1789760792.755694:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:0.0:1789760792.755698:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.755701:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760792.755705:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760792.755708:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760792.755716:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760792.755724:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760792.755744:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d300 00000008:00100000:0.0:1789760792.755907:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 74936320<>74940416 00000100:00100000:1.0:1789760792.756116:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565572992:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.767304:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985603 gen 3 00000008:00010000:1.0:1789760792.767422:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760792.767431:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.767442:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.767596:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760792.767628:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565572992:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760792.767955:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18295, 18295] 00000080:00200000:0.0:1789760792.768086:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760792.768092:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.768200:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74940416 i_blocks 146392 00000020:00200000:1.0:1789760792.768205:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.768217:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.768253:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74940416, count: 4096 00000080:00200000:1.0:1789760792.768261:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74940416, bytes: 4096 00000080:00200000:1.0:1789760792.768272:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.768280:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.768282:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74940416, 74944512) 0 0 00000080:00200000:1.0:1789760792.768287:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.768305:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18296, 18296] 00020000:00200000:1.0:1789760792.768313:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74940416 chunk: [74940416, 74944512) 74944512, 4096 00020000:00200000:1.0:1789760792.768318:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.768411:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18296, 18296] 00000008:00010000:1.0:1789760792.768430:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.768436:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.768447:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74940416, 74944512) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.768577:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74940416=0x4778000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.768640:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.768662:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74944512 00000020:00010000:1.0:1789760792.768671:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.768674:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.768678:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.768682:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.768685:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.768692:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.768698:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.768715:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760792.768919:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 74940416<>74944512 00000100:00100000:0.0:1789760792.769007:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565573120:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760792.785419:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985604 gen 3 00000008:00010000:0.0:1789760792.785477:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760792.785485:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760792.785496:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760792.785677:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760792.785709:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565573120:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760792.785864:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18296, 18296] 00000080:00200000:1.0:1789760792.785880:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760792.785887:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.785906:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74944512 i_blocks 146400 00000020:00200000:1.0:1789760792.785911:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.785921:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.785953:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74944512, count: 4096 00000080:00200000:1.0:1789760792.785961:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74944512, bytes: 4096 00000080:00200000:1.0:1789760792.785971:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.785978:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.785980:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74944512, 74948608) 0 0 00000080:00200000:1.0:1789760792.785985:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.786003:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18297, 18297] 00020000:00200000:1.0:1789760792.786012:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74944512 chunk: [74944512, 74948608) 74948608, 4096 00020000:00200000:1.0:1789760792.786016:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.786034:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18297, 18297] 00000008:00010000:1.0:1789760792.786064:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.786070:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.786082:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74944512, 74948608) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.786095:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74944512=0x4779000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.786321:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.786325:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74948608 00000020:00010000:1.0:1789760792.786334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.786337:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.786341:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.786345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.786348:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.786355:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.786362:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.786378:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760792.786624:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 74944512<>74948608 00000100:00100000:0.0:1789760792.786781:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565573248:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.795999:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985605 gen 3 00000008:00010000:1.0:1789760792.796050:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760792.796058:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.796068:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.796152:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760792.796182:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565573248:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.796438:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18297, 18297] 00000080:00200000:2.0:1789760792.796455:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.796460:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.796474:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74948608 i_blocks 146408 00000020:00200000:2.0:1789760792.796478:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.796552:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.796589:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74948608, count: 4096 00000080:00200000:2.0:1789760792.796597:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74948608, bytes: 4096 00000080:00200000:2.0:1789760792.796607:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.796616:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.796618:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74948608, 74952704) 0 0 00000080:00200000:2.0:1789760792.796622:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.796640:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18298, 18298] 00020000:00200000:2.0:1789760792.796763:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74948608 chunk: [74948608, 74952704) 74952704, 4096 00020000:00200000:2.0:1789760792.796768:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.796790:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18298, 18298] 00000008:00010000:2.0:1789760792.796808:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.796814:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.796826:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74948608, 74952704) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.796840:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74948608=0x477a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.796900:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.796903:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74952704 00000020:00010000:2.0:1789760792.796911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.796914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.796917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.796920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.796923:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.796930:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.796937:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.796954:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760792.797074:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 74948608<>74952704 00000100:00100000:3.0:1789760792.797164:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565573376:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760792.808634:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985606 gen 3 00000008:00010000:3.0:1789760792.808725:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760792.808733:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760792.808745:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760792.808768:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760792.808797:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565573376:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.808940:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18298, 18298] 00000080:00200000:2.0:1789760792.808957:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.808963:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.808978:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74952704 i_blocks 146416 00000020:00200000:2.0:1789760792.808982:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.808992:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.809023:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74952704, count: 4096 00000080:00200000:2.0:1789760792.809031:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74952704, bytes: 4096 00000080:00200000:2.0:1789760792.809040:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.809049:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.809051:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74952704, 74956800) 0 0 00000080:00200000:2.0:1789760792.809055:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.809072:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18299, 18299] 00020000:00200000:2.0:1789760792.809081:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74952704 chunk: [74952704, 74956800) 74956800, 4096 00020000:00200000:2.0:1789760792.809085:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.809104:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18299, 18299] 00000008:00010000:2.0:1789760792.809121:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.809127:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.809138:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74952704, 74956800) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.809152:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74952704=0x477b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.809205:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.809208:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74956800 00000020:00010000:2.0:1789760792.809216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.809315:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.809320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.809323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.809326:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.809334:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.809341:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.809358:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760792.809486:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 74952704<>74956800 00000100:00100000:3.0:1789760792.809602:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565573504:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760792.826335:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985607 gen 3 00000008:00010000:3.0:1789760792.826387:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760792.826396:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760792.826420:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760792.826443:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760792.826476:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565573504:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.826631:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18299, 18299] 00000080:00200000:2.0:1789760792.826687:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.826694:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.826710:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74956800 i_blocks 146424 00000020:00200000:2.0:1789760792.826713:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.826723:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.826759:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74956800, count: 4096 00000080:00200000:2.0:1789760792.826766:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74956800, bytes: 4096 00000080:00200000:2.0:1789760792.826777:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.826785:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.826787:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74956800, 74960896) 0 0 00000080:00200000:2.0:1789760792.826792:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.826809:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18300, 18300] 00020000:00200000:2.0:1789760792.826818:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74956800 chunk: [74956800, 74960896) 74960896, 4096 00020000:00200000:2.0:1789760792.826823:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.826842:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18300, 18300] 00000008:00010000:2.0:1789760792.826860:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.826866:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.826878:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74956800, 74960896) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.826891:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74956800=0x477c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.826948:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.826951:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74960896 00000020:00010000:2.0:1789760792.826959:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.826962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.826965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.826969:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.826972:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.826980:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.826986:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.827002:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760792.827120:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 74956800<>74960896 00000100:00100000:3.0:1789760792.827505:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760792.827524:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565573632:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760792.842476:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985608 gen 3 00000008:00010000:3.0:1789760792.842528:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:3.0:1789760792.842537:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760792.842548:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760792.842578:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:3.0:1789760792.842608:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565573632:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760792.842769:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18300, 18300] 00000080:00200000:1.0:1789760792.842788:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760792.842794:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.842810:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74960896 i_blocks 146432 00000020:00200000:1.0:1789760792.842814:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.842825:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.842859:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74960896, count: 4096 00000080:00200000:1.0:1789760792.842867:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74960896, bytes: 4096 00000080:00200000:1.0:1789760792.842877:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.842885:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.842888:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74960896, 74964992) 0 0 00000080:00200000:1.0:1789760792.842892:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.842909:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18301, 18301] 00020000:00200000:1.0:1789760792.842918:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74960896 chunk: [74960896, 74964992) 74964992, 4096 00020000:00200000:1.0:1789760792.842922:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.842943:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18301, 18301] 00000008:00010000:1.0:1789760792.842960:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.842966:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.842978:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74960896, 74964992) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.843332:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74960896=0x477d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.843468:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.843471:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74964992 00000020:00010000:1.0:1789760792.843481:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.843485:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.843489:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.843492:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.843495:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.843504:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.843512:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.843534:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760792.843695:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 74960896<>74964992 00000100:00100000:0.0:1789760792.843781:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565573760:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.864541:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985609 gen 3 00000008:00010000:1.0:1789760792.864595:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760792.864605:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.864615:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.864740:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760792.864770:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565573760:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.864940:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18301, 18301] 00000080:00200000:2.0:1789760792.864961:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.864968:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.864984:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74964992 i_blocks 146440 00000020:00200000:2.0:1789760792.864989:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.865000:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.865036:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74964992, count: 4096 00000080:00200000:2.0:1789760792.865044:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74964992, bytes: 4096 00000080:00200000:2.0:1789760792.865057:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.865066:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.865069:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74964992, 74969088) 0 0 00000080:00200000:2.0:1789760792.865074:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.865092:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18302, 18302] 00020000:00200000:2.0:1789760792.865100:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74964992 chunk: [74964992, 74969088) 74969088, 4096 00020000:00200000:2.0:1789760792.865105:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.865125:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18302, 18302] 00000008:00010000:2.0:1789760792.865143:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.865149:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.865161:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74964992, 74969088) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.865175:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74964992=0x477e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.865236:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.865239:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74969088 00000020:00010000:2.0:1789760792.865247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.865251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.865254:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.865258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.865261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.865268:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.865276:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.865295:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760792.865704:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 74964992<>74969088 00000100:00100000:3.0:1789760792.865930:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565573888:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760792.877112:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985610 gen 3 00000008:00010000:3.0:1789760792.877171:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760792.877180:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760792.877189:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760792.877215:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760792.877248:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565573888:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.877443:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18302, 18302] 00000080:00200000:2.0:1789760792.877460:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.877466:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.877482:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74969088 i_blocks 146448 00000020:00200000:2.0:1789760792.877494:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.877505:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.877542:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74969088, count: 4096 00000080:00200000:2.0:1789760792.877549:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74969088, bytes: 4096 00000080:00200000:2.0:1789760792.877560:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.877569:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.877571:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74969088, 74973184) 0 0 00000080:00200000:2.0:1789760792.877575:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.877594:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18303, 18303] 00020000:00200000:2.0:1789760792.877603:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74969088 chunk: [74969088, 74973184) 74973184, 4096 00020000:00200000:2.0:1789760792.877608:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.877628:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18303, 18303] 00000008:00010000:2.0:1789760792.877669:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.877675:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.877686:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74969088, 74973184) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.877701:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74969088=0x477f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.877753:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.877756:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74973184 00000020:00010000:2.0:1789760792.877764:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.877768:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.877770:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.877774:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.877777:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.877785:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.877792:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.877813:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760792.877936:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 74969088<>74973184 00000100:00100000:3.0:1789760792.878570:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760792.878594:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565574016:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760792.889854:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985611 gen 3 00000008:00010000:3.0:1789760792.889897:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760792.889905:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760792.889916:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760792.889937:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760792.889968:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565574016:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.890070:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18303, 18303] 00000080:00200000:2.0:1789760792.890086:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.890092:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.890105:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74973184 i_blocks 146456 00000020:00200000:2.0:1789760792.890108:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.890118:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.890149:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74973184, count: 4096 00000080:00200000:2.0:1789760792.890157:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74973184, bytes: 4096 00000080:00200000:2.0:1789760792.890165:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.890174:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.890177:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74973184, 74977280) 0 0 00000080:00200000:2.0:1789760792.890181:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.890197:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18304, 18304] 00020000:00200000:2.0:1789760792.890205:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74973184 chunk: [74973184, 74977280) 74977280, 4096 00020000:00200000:2.0:1789760792.890210:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.890227:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18304, 18304] 00000008:00010000:2.0:1789760792.890243:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.890249:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.890259:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74973184, 74977280) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.890271:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74973184=0x4780000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.890323:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.890326:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74977280 00000020:00010000:2.0:1789760792.890333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.890336:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.890340:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.890343:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.890346:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.890353:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.890360:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.890376:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760792.890514:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74973184<>74977280 00000100:00100000:3.0:1789760792.890605:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565574144:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.901696:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985612 gen 3 00000008:00010000:2.0:1789760792.901754:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:2.0:1789760792.901762:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.901774:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.901799:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:2.0:1789760792.901828:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565574144:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760792.901936:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18304, 18304] 00000080:00200000:1.0:1789760792.901962:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760792.901969:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.901985:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74977280 i_blocks 146464 00000020:00200000:1.0:1789760792.901989:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.901998:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.902031:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74977280, count: 4096 00000080:00200000:1.0:1789760792.902037:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74977280, bytes: 4096 00000080:00200000:1.0:1789760792.902059:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.902069:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.902071:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74977280, 74981376) 0 0 00000080:00200000:1.0:1789760792.902076:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.902093:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18305, 18305] 00020000:00200000:1.0:1789760792.902101:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74977280 chunk: [74977280, 74981376) 74981376, 4096 00020000:00200000:1.0:1789760792.902106:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.902124:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18305, 18305] 00000008:00010000:1.0:1789760792.902142:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.902148:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.902160:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74977280, 74981376) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.902174:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74977280=0x4781000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.902228:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.902230:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74981376 00000020:00010000:1.0:1789760792.902243:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.902247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.902250:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.902253:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.902256:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.902263:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.902269:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.902285:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760792.902396:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 74977280<>74981376 00000100:00100000:1.0:1789760792.902452:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760792.902467:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565574272:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.918093:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985613 gen 3 00000008:00010000:1.0:1789760792.918147:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760792.918156:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.918167:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.918198:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760792.918230:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565574272:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.918507:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18305, 18305] 00000080:00200000:2.0:1789760792.918525:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.918531:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.918547:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74981376 i_blocks 146472 00000020:00200000:2.0:1789760792.918552:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.918564:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.918598:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74981376, count: 4096 00000080:00200000:2.0:1789760792.918605:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74981376, bytes: 4096 00000080:00200000:2.0:1789760792.918619:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.918628:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.918630:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74981376, 74985472) 0 0 00000080:00200000:2.0:1789760792.918635:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.918670:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18306, 18306] 00020000:00200000:2.0:1789760792.918678:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74981376 chunk: [74981376, 74985472) 74985472, 4096 00020000:00200000:2.0:1789760792.918683:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.918702:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18306, 18306] 00000008:00010000:2.0:1789760792.918717:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.918723:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.918735:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74981376, 74985472) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.918758:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74981376=0x4782000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.918815:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.918818:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74985472 00000020:00010000:2.0:1789760792.918827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.918831:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.918834:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.918838:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.918841:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.918849:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.918856:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.918871:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760792.918983:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74981376<>74985472 00000100:00100000:2.0:1789760792.919035:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760792.919171:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565574400:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.932110:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985614 gen 3 00000008:00010000:2.0:1789760792.932157:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:2.0:1789760792.932164:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.932175:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.932243:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:2.0:1789760792.932275:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565574400:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760792.932546:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18306, 18306] 00000080:00200000:1.0:1789760792.932562:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760792.932568:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.932583:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74985472 i_blocks 146480 00000020:00200000:1.0:1789760792.932586:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.932597:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.932630:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74985472, count: 4096 00000080:00200000:1.0:1789760792.932637:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74985472, bytes: 4096 00000080:00200000:1.0:1789760792.932669:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.932678:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.932680:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74985472, 74989568) 0 0 00000080:00200000:1.0:1789760792.932684:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.932700:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18307, 18307] 00020000:00200000:1.0:1789760792.932709:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74985472 chunk: [74985472, 74989568) 74989568, 4096 00020000:00200000:1.0:1789760792.932714:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.932733:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18307, 18307] 00000008:00010000:1.0:1789760792.932813:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.932820:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.932832:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74985472, 74989568) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.932845:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74985472=0x4783000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.932898:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.932901:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74989568 00000020:00010000:1.0:1789760792.932908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.932912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.932915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.932918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.932922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.932928:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.932935:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.932952:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760792.933144:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 74985472<>74989568 00000100:00100000:0.0:1789760792.933255:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760792.933273:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565574528:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.949348:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985615 gen 3 00000008:00010000:1.0:1789760792.949624:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760792.949633:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.950030:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.950056:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760792.950093:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565574528:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.950229:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18307, 18307] 00000080:00200000:2.0:1789760792.950246:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.950252:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.950266:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74989568 i_blocks 146488 00000020:00200000:2.0:1789760792.950270:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.950279:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.950315:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74989568, count: 4096 00000080:00200000:2.0:1789760792.950323:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74989568, bytes: 4096 00000080:00200000:2.0:1789760792.950335:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.950354:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.950356:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74989568, 74993664) 0 0 00000080:00200000:2.0:1789760792.950361:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.950377:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18308, 18308] 00020000:00200000:2.0:1789760792.950386:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74989568 chunk: [74989568, 74993664) 74993664, 4096 00020000:00200000:2.0:1789760792.950392:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.950421:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18308, 18308] 00000008:00010000:2.0:1789760792.950440:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.950446:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.950459:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74989568, 74993664) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.950472:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74989568=0x4784000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.950533:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.950536:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74993664 00000020:00010000:2.0:1789760792.950544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.950548:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.950551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.950555:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.950558:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.950567:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.950574:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.950590:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760792.950732:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74989568<>74993664 00000100:00100000:3.0:1789760792.950809:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565574656:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.960289:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985616 gen 3 00000008:00010000:2.0:1789760792.960341:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:2.0:1789760792.960350:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.960360:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.960387:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:2.0:1789760792.960430:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565574656:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760792.960554:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18308, 18308] 00000080:00200000:1.0:1789760792.960570:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760792.960576:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.960592:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74993664 i_blocks 146496 00000020:00200000:1.0:1789760792.960596:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.960606:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.960678:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74993664, count: 4096 00000080:00200000:1.0:1789760792.960687:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74993664, bytes: 4096 00000080:00200000:1.0:1789760792.960697:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.960706:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.960708:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74993664, 74997760) 0 0 00000080:00200000:1.0:1789760792.960712:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.960729:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18309, 18309] 00020000:00200000:1.0:1789760792.960738:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74993664 chunk: [74993664, 74997760) 74997760, 4096 00020000:00200000:1.0:1789760792.960742:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.960760:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18309, 18309] 00000008:00010000:1.0:1789760792.960778:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.960784:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.960795:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74993664, 74997760) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.960809:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74993664=0x4785000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.960866:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.960869:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 74997760 00000020:00010000:1.0:1789760792.960878:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.960882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.960885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.960888:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.960891:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.960899:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.960906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.960922:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760792.961048:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 74993664<>74997760 00000100:00100000:0.0:1789760792.961172:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565574784:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760792.978386:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985617 gen 3 00000008:00010000:1.0:1789760792.978443:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760792.978451:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760792.978462:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760792.978488:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760792.978519:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565574784:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760792.978952:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18309, 18309] 00000080:00200000:2.0:1789760792.978970:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760792.978976:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760792.978993:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 74997760 i_blocks 146504 00000020:00200000:2.0:1789760792.978997:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760792.979008:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760792.979042:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 74997760, count: 4096 00000080:00200000:2.0:1789760792.979049:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 74997760, bytes: 4096 00000080:00200000:2.0:1789760792.979060:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760792.979069:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760792.979072:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [74997760, 75001856) 0 0 00000080:00200000:2.0:1789760792.979076:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760792.979093:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18310, 18310] 00020000:00200000:2.0:1789760792.979102:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 74997760 chunk: [74997760, 75001856) 75001856, 4096 00020000:00200000:2.0:1789760792.979107:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760792.979198:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18310, 18310] 00000008:00010000:2.0:1789760792.979217:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760792.979222:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760792.979245:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [74997760, 75001856) append 0 append lockpos 0 00000080:00200000:2.0:1789760792.979261:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=74997760=0x4786000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760792.979318:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760792.979321:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75001856 00000020:00010000:2.0:1789760792.979330:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760792.979333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.979336:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760792.979339:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760792.979342:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760792.979349:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760792.979356:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760792.979374:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760792.979506:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 74997760<>75001856 00000100:00100000:3.0:1789760792.979603:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565574912:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760792.995331:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985618 gen 3 00000008:00010000:2.0:1789760792.995385:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:2.0:1789760792.995394:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760792.995419:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760792.995444:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:2.0:1789760792.995476:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565574912:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760792.995598:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18310, 18310] 00000080:00200000:1.0:1789760792.995615:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760792.995621:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760792.995635:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75001856 i_blocks 146512 00000020:00200000:1.0:1789760792.995639:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760792.995680:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760792.995713:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75001856, count: 4096 00000080:00200000:1.0:1789760792.995721:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75001856, bytes: 4096 00000080:00200000:1.0:1789760792.995731:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760792.995739:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760792.995742:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75001856, 75005952) 0 0 00000080:00200000:1.0:1789760792.995746:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760792.995762:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18311, 18311] 00020000:00200000:1.0:1789760792.995770:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75001856 chunk: [75001856, 75005952) 75005952, 4096 00020000:00200000:1.0:1789760792.995774:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760792.995794:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18311, 18311] 00000008:00010000:1.0:1789760792.995812:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760792.995818:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760792.995830:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75001856, 75005952) append 0 append lockpos 0 00000080:00200000:1.0:1789760792.995842:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75001856=0x4787000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760792.995897:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760792.995900:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75005952 00000020:00010000:1.0:1789760792.995908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760792.995912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.995915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760792.995919:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760792.995922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760792.995929:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760792.995935:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760792.995953:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760792.996069:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75001856<>75005952 00000100:00100000:1.0:1789760792.996114:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760792.996140:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565575040:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.008935:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985619 gen 3 00000008:00010000:1.0:1789760793.008978:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760793.008985:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.008994:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.009268:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760793.009304:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565575040:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.010391:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18311, 18311] 00000080:00200000:2.0:1789760793.010422:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.010429:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.010505:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75005952 i_blocks 146520 00000020:00200000:2.0:1789760793.010511:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.010523:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.010558:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75005952, count: 4096 00000080:00200000:2.0:1789760793.010626:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75005952, bytes: 4096 00000080:00200000:2.0:1789760793.010642:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.010666:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.010669:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75005952, 75010048) 0 0 00000080:00200000:2.0:1789760793.010674:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.010691:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18312, 18312] 00020000:00200000:2.0:1789760793.010700:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75005952 chunk: [75005952, 75010048) 75010048, 4096 00020000:00200000:2.0:1789760793.010704:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.010726:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18312, 18312] 00000008:00010000:2.0:1789760793.010745:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.010752:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.010764:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75005952, 75010048) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.010781:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75005952=0x4788000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.010846:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.010849:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75010048 00000020:00010000:2.0:1789760793.010858:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.010862:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.010865:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.010868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.010871:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.010881:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.010887:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.010979:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760793.011109:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 75005952<>75010048 00000100:00100000:2.0:1789760793.011432:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760793.011524:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565575168:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.024259:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985620 gen 3 00000008:00010000:2.0:1789760793.024303:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:2.0:1789760793.024311:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.024320:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.024337:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:2.0:1789760793.024363:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565575168:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.024420:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18312, 18312] 00000080:00200000:3.0:1789760793.024439:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.024446:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760793.024462:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75010048 i_blocks 146528 00000020:00200000:3.0:1789760793.024466:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760793.024478:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760793.024512:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75010048, count: 4096 00000080:00200000:3.0:1789760793.024520:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75010048, bytes: 4096 00000080:00200000:3.0:1789760793.024530:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760793.024539:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760793.024542:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75010048, 75014144) 0 0 00000080:00200000:3.0:1789760793.024546:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760793.024562:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18313, 18313] 00020000:00200000:3.0:1789760793.024582:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75010048 chunk: [75010048, 75014144) 75014144, 4096 00020000:00200000:3.0:1789760793.024587:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760793.024611:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18313, 18313] 00000008:00010000:3.0:1789760793.024629:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760793.024635:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760793.024927:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75010048, 75014144) append 0 append lockpos 0 00000080:00200000:3.0:1789760793.024943:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75010048=0x4789000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760793.025001:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760793.025004:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75014144 00000020:00010000:3.0:1789760793.025013:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760793.025017:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.025020:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.025023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760793.025026:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760793.025034:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760793.025041:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760793.025060:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760793.025183:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 75010048<>75014144 00000100:00100000:3.0:1789760793.025357:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760793.025376:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565575296:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.034712:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985621 gen 3 00000008:00010000:3.0:1789760793.034993:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:3.0:1789760793.035003:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.035015:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.035043:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:3.0:1789760793.035075:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565575296:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.035458:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18313, 18313] 00000080:00200000:1.0:1789760793.035476:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.035482:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.035497:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75014144 i_blocks 146536 00000020:00200000:1.0:1789760793.035501:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.035512:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.035547:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75014144, count: 4096 00000080:00200000:1.0:1789760793.035554:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75014144, bytes: 4096 00000080:00200000:1.0:1789760793.035565:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.035574:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.035577:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75014144, 75018240) 0 0 00000080:00200000:1.0:1789760793.035582:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.035600:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18314, 18314] 00020000:00200000:1.0:1789760793.035608:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75014144 chunk: [75014144, 75018240) 75018240, 4096 00020000:00200000:1.0:1789760793.035613:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.035632:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18314, 18314] 00000008:00010000:1.0:1789760793.036229:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.036247:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.036266:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75014144, 75018240) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.036286:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75014144=0x478a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.036365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.036368:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75018240 00000020:00010000:1.0:1789760793.036377:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.036392:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.036395:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.036406:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.036410:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.036419:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.036426:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.036453:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760793.036592:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75014144<>75018240 00000100:00100000:0.0:1789760793.036692:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760793.036721:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565575424:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.054142:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985622 gen 3 00000008:00010000:1.0:1789760793.054192:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760793.054201:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.054219:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.054250:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760793.054285:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565575424:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.054689:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18314, 18314] 00000080:00200000:3.0:1789760793.054705:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.054712:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760793.054728:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75018240 i_blocks 146544 00000020:00200000:3.0:1789760793.054732:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760793.054742:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760793.054788:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75018240, count: 4096 00000080:00200000:3.0:1789760793.054796:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75018240, bytes: 4096 00000080:00200000:3.0:1789760793.054811:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760793.054820:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760793.054822:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75018240, 75022336) 0 0 00000080:00200000:3.0:1789760793.054827:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760793.054844:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18315, 18315] 00020000:00200000:3.0:1789760793.054854:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75018240 chunk: [75018240, 75022336) 75022336, 4096 00020000:00200000:3.0:1789760793.054859:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760793.054877:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18315, 18315] 00000008:00010000:3.0:1789760793.054896:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760793.054901:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760793.054912:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75018240, 75022336) append 0 append lockpos 0 00000080:00200000:3.0:1789760793.054926:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75018240=0x478b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760793.054986:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760793.054989:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75022336 00000020:00010000:3.0:1789760793.054997:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760793.055001:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.055004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.055009:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760793.055012:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760793.055019:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760793.055026:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760793.055043:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760793.055313:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 75018240<>75022336 00000100:00100000:2.0:1789760793.055832:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565575552:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.071920:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985623 gen 3 00000008:00010000:2.0:1789760793.072218:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760793.072228:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.072241:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.072311:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760793.072352:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565575552:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.073056:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18315, 18315] 00000080:00200000:3.0:1789760793.073075:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.073081:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760793.073097:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75022336 i_blocks 146552 00000020:00200000:3.0:1789760793.073101:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760793.073112:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760793.073145:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75022336, count: 4096 00000080:00200000:3.0:1789760793.073152:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75022336, bytes: 4096 00000080:00200000:3.0:1789760793.073165:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760793.073174:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760793.073177:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75022336, 75026432) 0 0 00000080:00200000:3.0:1789760793.073182:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760793.073199:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18316, 18316] 00020000:00200000:3.0:1789760793.073208:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75022336 chunk: [75022336, 75026432) 75026432, 4096 00020000:00200000:3.0:1789760793.073213:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760793.073233:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18316, 18316] 00000008:00010000:3.0:1789760793.073252:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760793.073258:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760793.073270:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75022336, 75026432) append 0 append lockpos 0 00000080:00200000:3.0:1789760793.073283:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75022336=0x478c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760793.073580:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760793.073584:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75026432 00000020:00010000:3.0:1789760793.073594:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760793.073597:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.073599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.073604:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760793.073607:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760793.073617:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760793.073624:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760793.073674:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760793.073868:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 75022336<>75026432 00000100:00100000:2.0:1789760793.074085:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565575680:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.090626:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985624 gen 3 00000008:00010000:2.0:1789760793.090709:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760793.090717:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.090726:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.090745:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760793.090769:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565575680:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.090979:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18316, 18316] 00000080:00200000:3.0:1789760793.090994:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.091000:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760793.091016:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75026432 i_blocks 146560 00000020:00200000:3.0:1789760793.091019:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760793.091029:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760793.091062:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75026432, count: 4096 00000080:00200000:3.0:1789760793.091070:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75026432, bytes: 4096 00000080:00200000:3.0:1789760793.091080:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760793.091089:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760793.091091:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75026432, 75030528) 0 0 00000080:00200000:3.0:1789760793.091096:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760793.091112:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18317, 18317] 00020000:00200000:3.0:1789760793.091121:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75026432 chunk: [75026432, 75030528) 75030528, 4096 00020000:00200000:3.0:1789760793.091126:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760793.091146:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18317, 18317] 00000008:00010000:3.0:1789760793.091163:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760793.091169:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760793.091180:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75026432, 75030528) append 0 append lockpos 0 00000080:00200000:3.0:1789760793.091193:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75026432=0x478d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760793.091249:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760793.091252:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75030528 00000020:00010000:3.0:1789760793.091261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760793.091263:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.091266:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.091270:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760793.091272:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760793.091279:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760793.091285:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760793.091302:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760793.092171:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 75026432<>75030528 00000100:00100000:2.0:1789760793.092226:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760793.092407:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565575808:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.103686:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985625 gen 3 00000008:00010000:2.0:1789760793.103737:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760793.103746:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.103757:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.103787:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760793.103817:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565575808:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.104003:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18317, 18317] 00000080:00200000:1.0:1789760793.104021:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.104027:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.104054:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75030528 i_blocks 146568 00000020:00200000:1.0:1789760793.104059:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.104069:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.104105:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75030528, count: 4096 00000080:00200000:1.0:1789760793.104113:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75030528, bytes: 4096 00000080:00200000:1.0:1789760793.104123:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.104132:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.104134:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75030528, 75034624) 0 0 00000080:00200000:1.0:1789760793.104138:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.104154:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18318, 18318] 00020000:00200000:1.0:1789760793.104162:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75030528 chunk: [75030528, 75034624) 75034624, 4096 00020000:00200000:1.0:1789760793.104167:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.104186:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18318, 18318] 00000008:00010000:1.0:1789760793.104203:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.104209:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.104220:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75030528, 75034624) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.104233:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75030528=0x478e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.104296:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.104299:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75034624 00000020:00010000:1.0:1789760793.104307:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.104311:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.104314:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.104318:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.104320:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.104327:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.104333:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.104349:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760793.104477:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75030528<>75034624 00000100:00100000:0.0:1789760793.104720:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565575936:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760793.114806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760793.122380:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985626 gen 3 00000008:00010000:1.0:1789760793.122442:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760793.122451:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.123008:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.123143:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760793.123178:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565575936:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.123285:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18318, 18318] 00000080:00200000:2.0:1789760793.123303:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.123310:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.123326:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75034624 i_blocks 146576 00000020:00200000:2.0:1789760793.123331:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.123341:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.123374:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75034624, count: 4096 00000080:00200000:2.0:1789760793.123382:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75034624, bytes: 4096 00000080:00200000:2.0:1789760793.123394:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.123414:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.123417:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75034624, 75038720) 0 0 00000080:00200000:2.0:1789760793.123422:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.123440:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18319, 18319] 00020000:00200000:2.0:1789760793.123449:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75034624 chunk: [75034624, 75038720) 75038720, 4096 00020000:00200000:2.0:1789760793.123454:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.123682:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18319, 18319] 00000008:00010000:2.0:1789760793.123701:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.123707:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.123718:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75034624, 75038720) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.123733:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75034624=0x478f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.123793:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.123795:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75038720 00000020:00010000:2.0:1789760793.123803:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.123807:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.123810:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.123813:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.123930:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.123942:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.123949:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.123967:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760793.125198:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 75034624<>75038720 00000100:00100000:2.0:1789760793.125275:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565576064:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.140387:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985627 gen 3 00000008:00010000:2.0:1789760793.140450:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:2.0:1789760793.140590:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.140604:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.140630:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:2.0:1789760793.140696:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565576064:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.140805:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18319, 18319] 00000080:00200000:3.0:1789760793.140892:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.140899:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760793.140914:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75038720 i_blocks 146584 00000020:00200000:3.0:1789760793.140917:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760793.140927:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760793.140960:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75038720, count: 4096 00000080:00200000:3.0:1789760793.140968:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75038720, bytes: 4096 00000080:00200000:3.0:1789760793.140979:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760793.140987:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760793.140990:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75038720, 75042816) 0 0 00000080:00200000:3.0:1789760793.140995:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760793.141013:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18320, 18320] 00020000:00200000:3.0:1789760793.141022:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75038720 chunk: [75038720, 75042816) 75042816, 4096 00020000:00200000:3.0:1789760793.141027:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760793.141147:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18320, 18320] 00000008:00010000:3.0:1789760793.141169:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760793.141175:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760793.141187:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75038720, 75042816) append 0 append lockpos 0 00000080:00200000:3.0:1789760793.141215:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75038720=0x4790000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760793.141280:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760793.141283:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75042816 00000020:00010000:3.0:1789760793.141291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760793.141295:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.141299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.141303:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760793.141306:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760793.141314:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760793.141320:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760793.141337:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760793.142364:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 75038720<>75042816 00000100:00100000:2.0:1789760793.142434:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760793.144429:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565576192:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.158548:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985628 gen 3 00000008:00010000:2.0:1789760793.158702:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760793.158711:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.158723:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.158754:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760793.158785:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565576192:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.159115:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18320, 18320] 00000080:00200000:1.0:1789760793.159133:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.159138:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.159153:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75042816 i_blocks 146592 00000020:00200000:1.0:1789760793.159157:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.159167:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.159553:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75042816, count: 4096 00000080:00200000:1.0:1789760793.159563:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75042816, bytes: 4096 00000080:00200000:1.0:1789760793.159677:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.159688:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.159691:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75042816, 75046912) 0 0 00000080:00200000:1.0:1789760793.159696:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.159714:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18321, 18321] 00020000:00200000:1.0:1789760793.159723:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75042816 chunk: [75042816, 75046912) 75046912, 4096 00020000:00200000:1.0:1789760793.159728:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.159752:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18321, 18321] 00000008:00010000:1.0:1789760793.159771:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.159777:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.159789:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75042816, 75046912) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.159804:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75042816=0x4791000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.159878:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.159881:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75046912 00000020:00010000:1.0:1789760793.159891:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.159894:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.159897:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.159900:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.159904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.159911:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.159918:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.159941:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760793.163950:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75042816<>75046912 00000100:00100000:1.0:1789760793.164061:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760793.164090:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565576320:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.184188:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985629 gen 3 00000008:00010000:1.0:1789760793.184294:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760793.184303:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.184314:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.184551:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760793.184585:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565576320:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.184901:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18321, 18321] 00000080:00200000:2.0:1789760793.184920:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.184926:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.184942:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75046912 i_blocks 146600 00000020:00200000:2.0:1789760793.184946:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.184957:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.184991:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75046912, count: 4096 00000080:00200000:2.0:1789760793.184998:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75046912, bytes: 4096 00000080:00200000:2.0:1789760793.185012:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.185021:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.185024:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75046912, 75051008) 0 0 00000080:00200000:2.0:1789760793.185029:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.185049:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18322, 18322] 00020000:00200000:2.0:1789760793.185058:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75046912 chunk: [75046912, 75051008) 75051008, 4096 00020000:00200000:2.0:1789760793.185063:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.185083:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18322, 18322] 00000008:00010000:2.0:1789760793.185104:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.185110:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.185121:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75046912, 75051008) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.185136:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75046912=0x4792000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.185265:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.185269:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75051008 00000020:00010000:2.0:1789760793.185279:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.185282:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.185286:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.185289:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.185291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.185299:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.185306:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.185324:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760793.185570:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 75046912<>75051008 00000100:00100000:3.0:1789760793.185702:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565576448:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.213079:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985630 gen 3 00000008:00010000:3.0:1789760793.213131:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:3.0:1789760793.213140:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.213151:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.213236:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:3.0:1789760793.213269:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565576448:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.213354:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18322, 18322] 00000080:00200000:1.0:1789760793.213371:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.213377:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.213392:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75051008 i_blocks 146608 00000020:00200000:1.0:1789760793.213396:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.213418:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.213452:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75051008, count: 4096 00000080:00200000:1.0:1789760793.213459:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75051008, bytes: 4096 00000080:00200000:1.0:1789760793.213468:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.213476:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.213479:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75051008, 75055104) 0 0 00000080:00200000:1.0:1789760793.213483:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.213501:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18323, 18323] 00020000:00200000:1.0:1789760793.213509:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75051008 chunk: [75051008, 75055104) 75055104, 4096 00020000:00200000:1.0:1789760793.213514:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.213532:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18323, 18323] 00000008:00010000:1.0:1789760793.213549:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.213555:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.213566:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75051008, 75055104) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.213580:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75051008=0x4793000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.213637:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.213640:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75055104 00000020:00010000:1.0:1789760793.213787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.213791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.213795:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.213799:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.213801:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.213809:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.213816:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.213833:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760793.215327:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75051008<>75055104 00000100:00100000:0.0:1789760793.215511:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565576576:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.230237:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985631 gen 3 00000008:00010000:1.0:1789760793.230287:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760793.230295:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.230306:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.230335:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000080:00200000:2.0:1789760793.230517:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18323, 18323] 00000080:00200000:2.0:1789760793.230536:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.230543:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.230557:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75055104 i_blocks 146616 00000020:00200000:2.0:1789760793.230561:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.230571:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.230604:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75055104, count: 4096 00000080:00200000:2.0:1789760793.230613:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75055104, bytes: 4096 00000080:00200000:2.0:1789760793.230626:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.230635:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.230637:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75055104, 75059200) 0 0 00000080:00200000:2.0:1789760793.230641:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.230675:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18324, 18324] 00020000:00200000:2.0:1789760793.230684:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75055104 chunk: [75055104, 75059200) 75059200, 4096 00020000:00200000:2.0:1789760793.230689:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.230708:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18324, 18324] 00000008:00010000:2.0:1789760793.230726:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.230732:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.230745:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75055104, 75059200) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.230759:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75055104=0x4794000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.230807:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.230811:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75059200 00000020:00010000:2.0:1789760793.230817:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.230821:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.230824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.230827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.230830:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.230837:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.230843:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1789760793.230974:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565576576:192.168.203.147@tcp:4:dd.0 00000008:00010000:1.0:1789760793.231024:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354ff00 00000008:00100000:1.0:1789760793.231136:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75055104<>75059200 00000100:00100000:0.0:1789760793.231200:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760793.231570:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565576704:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.244034:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985632 gen 3 00000008:00010000:1.0:1789760793.244082:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354ff00 00000020:00200000:1.0:1789760793.244091:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.244103:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.244128:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354ff00 00000100:00100000:1.0:1789760793.244158:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565576704:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.244285:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18324, 18324] 00000080:00200000:2.0:1789760793.244301:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.244307:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.244322:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75059200 i_blocks 146624 00000020:00200000:2.0:1789760793.244326:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.244336:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.244448:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75059200, count: 4096 00000080:00200000:2.0:1789760793.244523:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75059200, bytes: 4096 00000080:00200000:2.0:1789760793.244534:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.244542:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.244545:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75059200, 75063296) 0 0 00000080:00200000:2.0:1789760793.244549:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.244566:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18325, 18325] 00020000:00200000:2.0:1789760793.244574:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75059200 chunk: [75059200, 75063296) 75063296, 4096 00020000:00200000:2.0:1789760793.244579:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.244599:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18325, 18325] 00000008:00010000:2.0:1789760793.244617:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.244623:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.244634:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75059200, 75063296) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.244675:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75059200=0x4795000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.244731:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.244735:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75063296 00000020:00010000:2.0:1789760793.244743:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.244747:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.244750:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.244754:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.244757:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.244764:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.244771:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.244788:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f180 00000008:00100000:2.0:1789760793.245530:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 75059200<>75063296 00000100:00100000:3.0:1789760793.245795:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565576832:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.263355:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985633 gen 3 00000008:00010000:3.0:1789760793.263416:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f180 00000020:00200000:3.0:1789760793.263426:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.263437:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.263460:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f180 00000100:00100000:3.0:1789760793.263492:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565576832:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.263548:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18325, 18325] 00000080:00200000:2.0:1789760793.263567:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.263574:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.263588:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75063296 i_blocks 146632 00000020:00200000:2.0:1789760793.263592:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.263602:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.263635:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75063296, count: 4096 00000080:00200000:2.0:1789760793.264176:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75063296, bytes: 4096 00000080:00200000:2.0:1789760793.264191:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.264200:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.264203:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75063296, 75067392) 0 0 00000080:00200000:2.0:1789760793.264208:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.264225:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18326, 18326] 00020000:00200000:2.0:1789760793.264235:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75063296 chunk: [75063296, 75067392) 75067392, 4096 00020000:00200000:2.0:1789760793.264240:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.264261:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18326, 18326] 00000008:00010000:2.0:1789760793.264280:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.264286:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.264297:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75063296, 75067392) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.264310:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75063296=0x4796000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.264381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.264385:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75067392 00000020:00010000:2.0:1789760793.264393:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.264406:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.264410:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.264414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.264417:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.264425:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.264432:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.264462:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f180 00000008:00100000:2.0:1789760793.264584:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 75063296<>75067392 00000100:00100000:3.0:1789760793.264991:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760793.265015:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565576960:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.281557:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985634 gen 3 00000008:00010000:3.0:1789760793.281894:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f180 00000020:00200000:3.0:1789760793.281904:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.281918:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.281944:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f180 00000100:00100000:3.0:1789760793.281981:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565576960:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.282206:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18326, 18326] 00000080:00200000:2.0:1789760793.282223:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.282230:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.282247:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75067392 i_blocks 146640 00000020:00200000:2.0:1789760793.282251:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.282262:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.282299:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75067392, count: 4096 00000080:00200000:2.0:1789760793.282308:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75067392, bytes: 4096 00000080:00200000:2.0:1789760793.282323:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.282333:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.282336:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75067392, 75071488) 0 0 00000080:00200000:2.0:1789760793.282340:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.282438:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18327, 18327] 00020000:00200000:2.0:1789760793.282448:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75067392 chunk: [75067392, 75071488) 75071488, 4096 00020000:00200000:2.0:1789760793.282453:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.282474:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18327, 18327] 00000008:00010000:2.0:1789760793.282494:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.282500:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.282511:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75067392, 75071488) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.282526:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75067392=0x4797000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.282587:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.282591:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75071488 00000020:00010000:2.0:1789760793.282599:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.282602:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.282605:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.282608:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.282611:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.282618:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.282624:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.282641:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354ff00 00000008:00100000:2.0:1789760793.283538:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 75067392<>75071488 00000100:00100000:3.0:1789760793.283612:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565577088:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.300549:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985635 gen 3 00000008:00010000:2.0:1789760793.300599:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354ff00 00000020:00200000:2.0:1789760793.300609:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.300620:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.300678:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354ff00 00000100:00100000:2.0:1789760793.300709:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565577088:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.300888:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18327, 18327] 00000080:00200000:3.0:1789760793.300906:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.300913:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760793.300927:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75071488 i_blocks 146648 00000020:00200000:3.0:1789760793.300931:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760793.300941:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760793.300977:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75071488, count: 4096 00000080:00200000:3.0:1789760793.300985:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75071488, bytes: 4096 00000080:00200000:3.0:1789760793.300996:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760793.301006:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760793.301008:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75071488, 75075584) 0 0 00000080:00200000:3.0:1789760793.301013:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760793.301031:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18328, 18328] 00020000:00200000:3.0:1789760793.301040:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75071488 chunk: [75071488, 75075584) 75075584, 4096 00020000:00200000:3.0:1789760793.301045:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760793.301067:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18328, 18328] 00000008:00010000:3.0:1789760793.301085:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760793.301091:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760793.301104:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75071488, 75075584) append 0 append lockpos 0 00000080:00200000:3.0:1789760793.301118:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75071488=0x4798000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760793.301176:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760793.301179:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75075584 00000020:00010000:3.0:1789760793.301187:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:3.0:1789760793.301191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.301194:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.301198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760793.301201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760793.301209:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760793.301216:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760793.301233:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760793.301631:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 75071488<>75075584 00000100:00100000:2.0:1789760793.301957:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565577216:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.316368:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985636 gen 3 00000008:00010000:2.0:1789760793.316434:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760793.316442:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.316453:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.316481:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760793.316568:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565577216:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.316691:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18328, 18328] 00000080:00200000:1.0:1789760793.316707:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.316713:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.316727:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75075584 i_blocks 146656 00000020:00200000:1.0:1789760793.316730:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.316739:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.316772:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75075584, count: 4096 00000080:00200000:1.0:1789760793.316779:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75075584, bytes: 4096 00000080:00200000:1.0:1789760793.316789:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.316797:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.316799:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75075584, 75079680) 0 0 00000080:00200000:1.0:1789760793.316803:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.316821:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18329, 18329] 00020000:00200000:1.0:1789760793.316830:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75075584 chunk: [75075584, 75079680) 75079680, 4096 00020000:00200000:1.0:1789760793.316834:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.316853:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18329, 18329] 00000008:00010000:1.0:1789760793.316872:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.316878:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.316890:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75075584, 75079680) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.316903:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75075584=0x4799000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.316960:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.316963:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75079680 00000020:00010000:1.0:1789760793.316971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.316975:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.316978:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.316981:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.316985:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.316992:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.316999:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.317015:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760793.317137:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75075584<>75079680 00000100:00100000:0.0:1789760793.317214:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565577344:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.327703:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985637 gen 3 00000008:00010000:1.0:1789760793.327759:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760793.327767:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.327778:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.327804:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760793.327834:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565577344:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.328223:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18329, 18329] 00000080:00200000:2.0:1789760793.328240:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.328247:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.328262:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75079680 i_blocks 146664 00000020:00200000:2.0:1789760793.328267:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.328276:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.328311:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75079680, count: 4096 00000080:00200000:2.0:1789760793.328318:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75079680, bytes: 4096 00000080:00200000:2.0:1789760793.328392:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.328411:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.328413:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75079680, 75083776) 0 0 00000080:00200000:2.0:1789760793.328418:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.328437:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18330, 18330] 00020000:00200000:2.0:1789760793.328446:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75079680 chunk: [75079680, 75083776) 75083776, 4096 00020000:00200000:2.0:1789760793.328451:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.328473:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18330, 18330] 00000008:00010000:2.0:1789760793.328547:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.328554:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.328566:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75079680, 75083776) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.328580:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75079680=0x479a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.328637:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.328640:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75083776 00000020:00010000:2.0:1789760793.328735:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.328739:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.328742:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.328746:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.328749:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.328758:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.328765:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.328782:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354ff00 00000008:00100000:2.0:1789760793.328907:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 75079680<>75083776 00000100:00100000:3.0:1789760793.328981:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565577472:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.339764:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985638 gen 3 00000008:00010000:3.0:1789760793.339815:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354ff00 00000020:00200000:3.0:1789760793.339824:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.339834:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.339857:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354ff00 00000100:00100000:3.0:1789760793.339886:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565577472:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.339990:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18330, 18330] 00000080:00200000:2.0:1789760793.340006:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.340012:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.340027:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75083776 i_blocks 146672 00000020:00200000:2.0:1789760793.340032:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.340042:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.340077:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75083776, count: 4096 00000080:00200000:2.0:1789760793.340084:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75083776, bytes: 4096 00000080:00200000:2.0:1789760793.340093:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.340102:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.340104:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75083776, 75087872) 0 0 00000080:00200000:2.0:1789760793.340109:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.340126:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18331, 18331] 00020000:00200000:2.0:1789760793.340135:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75083776 chunk: [75083776, 75087872) 75087872, 4096 00020000:00200000:2.0:1789760793.340140:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.340158:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18331, 18331] 00000008:00010000:2.0:1789760793.340177:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.340183:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.340195:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75083776, 75087872) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.340207:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75083776=0x479b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.340262:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.340265:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75087872 00000020:00010000:2.0:1789760793.340273:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.340277:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.340280:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.340283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.340286:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.340293:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.340299:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.340313:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760793.340445:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 75083776<>75087872 00000100:00100000:3.0:1789760793.340694:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565577600:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.351726:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985639 gen 3 00000008:00010000:3.0:1789760793.351782:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:3.0:1789760793.351791:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.351803:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.351830:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:3.0:1789760793.351871:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565577600:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.352073:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18331, 18331] 00000080:00200000:2.0:1789760793.352091:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.352098:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.352115:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75087872 i_blocks 146680 00000020:00200000:2.0:1789760793.352120:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.352133:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.352175:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75087872, count: 4096 00000080:00200000:2.0:1789760793.352185:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75087872, bytes: 4096 00000080:00200000:2.0:1789760793.352198:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.352209:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.352212:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75087872, 75091968) 0 0 00000080:00200000:2.0:1789760793.352216:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.352239:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18332, 18332] 00020000:00200000:2.0:1789760793.352250:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75087872 chunk: [75087872, 75091968) 75091968, 4096 00020000:00200000:2.0:1789760793.352255:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.352280:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18332, 18332] 00000008:00010000:2.0:1789760793.352302:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.352308:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.352321:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75087872, 75091968) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.352337:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75087872=0x479c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.352411:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.352414:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75091968 00000020:00010000:2.0:1789760793.352421:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.352425:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.352429:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.352432:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.352436:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.352444:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.352451:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.352470:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760793.352598:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 75087872<>75091968 00000100:00100000:3.0:1789760793.352886:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565577728:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.367170:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985640 gen 3 00000008:00010000:2.0:1789760793.367220:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760793.367229:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.367240:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.367327:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760793.367370:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565577728:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.367919:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18332, 18332] 00000080:00200000:1.0:1789760793.367937:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.367943:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.367958:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75091968 i_blocks 146688 00000020:00200000:1.0:1789760793.367962:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.367973:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.368009:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75091968, count: 4096 00000080:00200000:1.0:1789760793.368018:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75091968, bytes: 4096 00000080:00200000:1.0:1789760793.368031:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.368040:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.368042:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75091968, 75096064) 0 0 00000080:00200000:1.0:1789760793.368046:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.368064:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18333, 18333] 00020000:00200000:1.0:1789760793.368072:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75091968 chunk: [75091968, 75096064) 75096064, 4096 00020000:00200000:1.0:1789760793.368077:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.368097:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18333, 18333] 00000008:00010000:1.0:1789760793.368116:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.368123:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.368135:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75091968, 75096064) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.368150:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75091968=0x479d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.368215:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.368218:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75096064 00000020:00010000:1.0:1789760793.368227:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.368231:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.368233:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.368237:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.368240:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.368248:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.368254:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.368278:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760793.368871:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75091968<>75096064 00000100:00100000:0.0:1789760793.368951:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565577856:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.378201:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985641 gen 3 00000008:00010000:1.0:1789760793.378259:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760793.378270:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.378282:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.378310:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000080:00200000:2.0:1789760793.378393:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18333, 18333] 00000080:00200000:2.0:1789760793.378421:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.378428:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.378443:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75096064 i_blocks 146696 00000020:00200000:2.0:1789760793.378447:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.378457:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.378507:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75096064, count: 4096 00000080:00200000:2.0:1789760793.378517:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75096064, bytes: 4096 00000080:00200000:2.0:1789760793.378528:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.378537:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.378539:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75096064, 75100160) 0 0 00000080:00200000:2.0:1789760793.378544:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.378583:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18334, 18334] 00020000:00200000:2.0:1789760793.378592:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75096064 chunk: [75096064, 75100160) 75100160, 4096 00020000:00200000:2.0:1789760793.378601:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.378625:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18334, 18334] 00000008:00010000:2.0:1789760793.378678:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.378685:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.378697:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75096064, 75100160) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.378712:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75096064=0x479e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.378785:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.378788:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75100160 00000020:00010000:2.0:1789760793.378797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.378801:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.378804:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.378807:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.378810:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.378817:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.378825:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1789760793.379228:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565577856:192.168.203.147@tcp:4:dd.0 00000008:00010000:1.0:1789760793.379280:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:1.0:1789760793.379470:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75096064<>75100160 00000100:00100000:1.0:1789760793.379520:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565577984:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.391016:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985642 gen 3 00000008:00010000:1.0:1789760793.391067:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:1.0:1789760793.391078:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.391089:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.391113:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:1.0:1789760793.391148:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565577984:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.391895:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18334, 18334] 00000080:00200000:2.0:1789760793.391915:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.391921:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.391938:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75100160 i_blocks 146704 00000020:00200000:2.0:1789760793.391942:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.391951:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.391981:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75100160, count: 4096 00000080:00200000:2.0:1789760793.391987:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75100160, bytes: 4096 00000080:00200000:2.0:1789760793.391996:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.392003:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.392005:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75100160, 75104256) 0 0 00000080:00200000:2.0:1789760793.392008:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.392124:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18335, 18335] 00020000:00200000:2.0:1789760793.392133:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75100160 chunk: [75100160, 75104256) 75104256, 4096 00020000:00200000:2.0:1789760793.392136:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.392157:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18335, 18335] 00000008:00010000:2.0:1789760793.392178:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.392366:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.392378:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75100160, 75104256) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.392392:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75100160=0x479f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.392467:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.392470:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75104256 00000020:00010000:2.0:1789760793.392479:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.392483:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.392486:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.392489:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.392492:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.392639:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.392795:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.392820:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760793.392957:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 75100160<>75104256 00000100:00100000:2.0:1789760793.393016:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760793.393038:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565578112:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.403157:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985643 gen 3 00000008:00010000:2.0:1789760793.403251:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760793.403259:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.403271:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.403450:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760793.403482:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565578112:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.403894:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18335, 18335] 00000080:00200000:1.0:1789760793.403980:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.403987:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.404003:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75104256 i_blocks 146712 00000020:00200000:1.0:1789760793.404006:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.404017:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.404053:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75104256, count: 4096 00000080:00200000:1.0:1789760793.404061:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75104256, bytes: 4096 00000080:00200000:1.0:1789760793.404073:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.404081:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.404084:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75104256, 75108352) 0 0 00000080:00200000:1.0:1789760793.404089:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.404106:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18336, 18336] 00020000:00200000:1.0:1789760793.404114:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75104256 chunk: [75104256, 75108352) 75108352, 4096 00020000:00200000:1.0:1789760793.404119:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.404139:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18336, 18336] 00000008:00010000:1.0:1789760793.404157:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.404163:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.404175:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75104256, 75108352) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.404190:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75104256=0x47a0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.404301:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.404304:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75108352 00000020:00010000:1.0:1789760793.404435:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.404439:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.404443:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.404446:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0998 (ffff93f84418ae18/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.404449:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418ae18/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.404457:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.404464:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.404545:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760793.404706:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75104256<>75108352 00000100:00100000:1.0:1789760793.404899:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760793.405027:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565578240:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.419250:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985644 gen 3 00000008:00010000:1.0:1789760793.419297:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:1.0:1789760793.419306:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.419315:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.419335:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:1.0:1789760793.419363:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565578240:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760793.419676:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18336, 18336] 00000080:00200000:0.0:1789760793.419696:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760793.419703:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760793.419733:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75108352 i_blocks 146720 00000020:00200000:0.0:1789760793.419746:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760793.419756:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760793.419815:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75108352, count: 4096 00000080:00200000:0.0:1789760793.419823:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75108352, bytes: 4096 00000080:00200000:0.0:1789760793.419833:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760793.419841:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760793.419844:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75108352, 75112448) 0 0 00000080:00200000:0.0:1789760793.419848:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760793.419865:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18337, 18337] 00020000:00200000:0.0:1789760793.419875:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75108352 chunk: [75108352, 75112448) 75112448, 4096 00020000:00200000:0.0:1789760793.419879:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760793.419899:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18337, 18337] 00000008:00010000:0.0:1789760793.420074:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760793.420082:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760793.420094:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75108352, 75112448) append 0 append lockpos 0 00000080:00200000:0.0:1789760793.420109:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75108352=0x47a1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760793.420166:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760793.420169:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75112448 00000020:00010000:0.0:1789760793.420176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:0.0:1789760793.420179:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760793.420183:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760793.420186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760793.420189:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760793.420196:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760793.420202:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760793.420221:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760793.420339:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 75108352<>75112448 00000100:00100000:1.0:1789760793.420498:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565578368:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.429451:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985645 gen 3 00000008:00010000:1.0:1789760793.429516:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760793.429524:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.429535:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.429564:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:1.0:1789760793.429596:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565578368:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.430044:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18337, 18337] 00000080:00200000:2.0:1789760793.430061:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.430067:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.430087:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75112448 i_blocks 146728 00000020:00200000:2.0:1789760793.430092:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.430104:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.430141:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75112448, count: 4096 00000080:00200000:2.0:1789760793.430149:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75112448, bytes: 4096 00000080:00200000:2.0:1789760793.430161:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.430169:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.430172:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75112448, 75116544) 0 0 00000080:00200000:2.0:1789760793.430176:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.430193:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18338, 18338] 00020000:00200000:2.0:1789760793.430202:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75112448 chunk: [75112448, 75116544) 75116544, 4096 00020000:00200000:2.0:1789760793.430207:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.430228:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18338, 18338] 00000008:00010000:2.0:1789760793.430248:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.430255:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.430267:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75112448, 75116544) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.430283:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75112448=0x47a2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.430351:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.430356:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75116544 00000020:00010000:2.0:1789760793.430364:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.430368:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.430371:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.430375:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.430378:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.430427:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.430434:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.430450:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760793.431345:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 75112448<>75116544 00000100:00100000:3.0:1789760793.431675:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760793.431698:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565578496:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.442081:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985646 gen 3 00000008:00010000:3.0:1789760793.442131:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760793.442140:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.442153:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.442175:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760793.442205:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565578496:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.442459:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18338, 18338] 00000080:00200000:2.0:1789760793.442476:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.442482:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.442566:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75116544 i_blocks 146736 00000020:00200000:2.0:1789760793.442571:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.442581:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.442617:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75116544, count: 4096 00000080:00200000:2.0:1789760793.442625:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75116544, bytes: 4096 00000080:00200000:2.0:1789760793.442637:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.442680:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.442683:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75116544, 75120640) 0 0 00000080:00200000:2.0:1789760793.442687:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.442705:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18339, 18339] 00020000:00200000:2.0:1789760793.443163:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75116544 chunk: [75116544, 75120640) 75120640, 4096 00020000:00200000:2.0:1789760793.443171:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.443194:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18339, 18339] 00000008:00010000:2.0:1789760793.443214:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.443220:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.443233:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75116544, 75120640) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.443247:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75116544=0x47a3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.443310:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.443314:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75120640 00000020:00010000:2.0:1789760793.443323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.443327:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.443329:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.443333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.443336:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.443344:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.443361:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.443394:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760793.443977:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 75116544<>75120640 00000100:00100000:3.0:1789760793.444096:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760793.444122:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565578624:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.454498:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985647 gen 3 00000008:00010000:3.0:1789760793.454538:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:3.0:1789760793.454545:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.454554:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.454579:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:3.0:1789760793.454605:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565578624:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.455853:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18339, 18339] 00000080:00200000:2.0:1789760793.455870:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.455876:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.455888:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75120640 i_blocks 146744 00000020:00200000:2.0:1789760793.455892:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.455903:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.455943:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75120640, count: 4096 00000080:00200000:2.0:1789760793.455951:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75120640, bytes: 4096 00000080:00200000:2.0:1789760793.455962:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.455969:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.455972:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75120640, 75124736) 0 0 00000080:00200000:2.0:1789760793.455975:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.455988:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18340, 18340] 00020000:00200000:2.0:1789760793.455995:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75120640 chunk: [75120640, 75124736) 75124736, 4096 00020000:00200000:2.0:1789760793.455999:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.456016:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18340, 18340] 00000008:00010000:2.0:1789760793.456031:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.456036:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.456046:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75120640, 75124736) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.456056:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75120640=0x47a4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.456106:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.456110:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75124736 00000020:00010000:2.0:1789760793.456117:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.456120:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.456858:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.456863:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.456866:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.456876:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.456883:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.456903:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760793.457117:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 75120640<>75124736 00000100:00100000:3.0:1789760793.457665:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760793.457687:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565578752:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.470105:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985648 gen 3 00000008:00010000:2.0:1789760793.470263:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760793.470271:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.470278:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.470956:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760793.470982:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565578752:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.471696:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18340, 18340] 00000080:00200000:1.0:1789760793.471712:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.471718:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.471730:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75124736 i_blocks 146752 00000020:00200000:1.0:1789760793.471734:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.472028:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.472057:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75124736, count: 4096 00000080:00200000:1.0:1789760793.472064:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75124736, bytes: 4096 00000080:00200000:1.0:1789760793.472074:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.472081:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.472083:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75124736, 75128832) 0 0 00000080:00200000:1.0:1789760793.472087:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.472100:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18341, 18341] 00020000:00200000:1.0:1789760793.472107:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75124736 chunk: [75124736, 75128832) 75128832, 4096 00020000:00200000:1.0:1789760793.472112:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.472197:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18341, 18341] 00000008:00010000:1.0:1789760793.472212:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.472217:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.472226:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75124736, 75128832) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.472237:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75124736=0x47a5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.472279:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.472282:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75128832 00000020:00010000:1.0:1789760793.472407:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.472412:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.472415:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.472418:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.472420:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.472426:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.472432:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.472901:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:2.0:1789760793.473019:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 75124736<>75128832 00000100:00100000:2.0:1789760793.473077:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565578880:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.490232:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985649 gen 3 00000008:00010000:2.0:1789760793.490267:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:2.0:1789760793.490273:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.490282:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.490365:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:2.0:1789760793.490390:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565578880:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.493744:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18341, 18341] 00000080:00200000:1.0:1789760793.493828:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.493834:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.493848:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75128832 i_blocks 146760 00000020:00200000:1.0:1789760793.493852:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.493864:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.493891:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75128832, count: 4096 00000080:00200000:1.0:1789760793.493897:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75128832, bytes: 4096 00000080:00200000:1.0:1789760793.493964:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.493973:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.493975:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75128832, 75132928) 0 0 00000080:00200000:1.0:1789760793.493979:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.493992:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18342, 18342] 00020000:00200000:1.0:1789760793.494007:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75128832 chunk: [75128832, 75132928) 75132928, 4096 00020000:00200000:1.0:1789760793.494011:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.494028:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18342, 18342] 00000008:00010000:1.0:1789760793.494043:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.494048:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.494067:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75128832, 75132928) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.494078:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75128832=0x47a6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.494128:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.494130:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75132928 00000020:00010000:1.0:1789760793.494137:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.494139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.494142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.494144:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.494147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.494153:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.494158:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.494178:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.494274:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75128832<>75132928 00000100:00100000:1.0:1789760793.494338:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760793.494357:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565579008:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.506477:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985650 gen 3 00000008:00010000:1.0:1789760793.506516:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.506523:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.506531:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.507314:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760793.507343:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565579008:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.508663:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18342, 18342] 00000080:00200000:2.0:1789760793.508680:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.508685:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.508698:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75132928 i_blocks 146768 00000020:00200000:2.0:1789760793.508702:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.508714:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.508739:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75132928, count: 4096 00000080:00200000:2.0:1789760793.508745:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75132928, bytes: 4096 00000080:00200000:2.0:1789760793.508756:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.508832:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.508834:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75132928, 75137024) 0 0 00000080:00200000:2.0:1789760793.509236:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.509252:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18343, 18343] 00020000:00200000:2.0:1789760793.509259:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75132928 chunk: [75132928, 75137024) 75137024, 4096 00020000:00200000:2.0:1789760793.509263:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.509291:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18343, 18343] 00000008:00010000:2.0:1789760793.509305:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.509310:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.509319:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75132928, 75137024) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.509329:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75132928=0x47a7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.509381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.509384:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75137024 00000020:00010000:2.0:1789760793.509390:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.509393:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.509410:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.509413:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.509416:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.509422:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.509428:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.509451:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760793.509557:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 75132928<>75137024 00000100:00100000:3.0:1789760793.509677:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565579136:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.523742:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985651 gen 3 00000008:00010000:3.0:1789760793.523782:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760793.523788:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.523797:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.523822:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760793.523850:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565579136:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.524075:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18343, 18343] 00000080:00200000:2.0:1789760793.524088:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.524094:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.524105:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75137024 i_blocks 146776 00000020:00200000:2.0:1789760793.524109:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.524117:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.524142:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75137024, count: 4096 00000080:00200000:2.0:1789760793.524148:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75137024, bytes: 4096 00000080:00200000:2.0:1789760793.524157:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.524164:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.524166:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75137024, 75141120) 0 0 00000080:00200000:2.0:1789760793.524169:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.524181:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18344, 18344] 00020000:00200000:2.0:1789760793.524188:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75137024 chunk: [75137024, 75141120) 75141120, 4096 00020000:00200000:2.0:1789760793.524191:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.524205:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18344, 18344] 00000008:00010000:2.0:1789760793.524217:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.524221:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.524229:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75137024, 75141120) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.524239:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75137024=0x47a8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.524276:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.524278:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75141120 00000020:00010000:2.0:1789760793.524284:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.524287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.524290:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.524293:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.524296:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.524301:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.524306:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.524318:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760793.524422:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 75137024<>75141120 00000100:00100000:3.0:1789760793.524524:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565579264:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.535713:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985652 gen 3 00000008:00010000:3.0:1789760793.535760:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760793.535767:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.535776:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.535794:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760793.535817:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565579264:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.535823:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18344, 18344] 00000080:00200000:2.0:1789760793.535836:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.535841:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.535853:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75141120 i_blocks 146784 00000020:00200000:2.0:1789760793.535857:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.535864:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.535888:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75141120, count: 4096 00000080:00200000:2.0:1789760793.535894:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75141120, bytes: 4096 00000080:00200000:2.0:1789760793.535902:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.535980:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.535983:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75141120, 75145216) 0 0 00000080:00200000:2.0:1789760793.535987:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.536001:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18345, 18345] 00020000:00200000:2.0:1789760793.536008:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75141120 chunk: [75141120, 75145216) 75145216, 4096 00020000:00200000:2.0:1789760793.536012:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.536027:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18345, 18345] 00000008:00010000:2.0:1789760793.536041:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.536045:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.536053:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75141120, 75145216) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.536064:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75141120=0x47a9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.536109:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.536112:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75145216 00000020:00010000:2.0:1789760793.536120:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.536123:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.536126:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.536129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.536132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.536138:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.536144:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.536158:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760793.536337:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 75141120<>75145216 00000100:00100000:3.0:1789760793.536878:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760793.536896:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565579392:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.544679:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985653 gen 3 00000008:00010000:3.0:1789760793.544733:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760793.544752:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.544763:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.544781:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760793.544811:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565579392:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.545151:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18345, 18345] 00000080:00200000:2.0:1789760793.545172:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.545178:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.545192:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75145216 i_blocks 146792 00000020:00200000:2.0:1789760793.545197:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.545207:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.545231:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75145216, count: 4096 00000080:00200000:2.0:1789760793.545239:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75145216, bytes: 4096 00000080:00200000:2.0:1789760793.545249:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.545257:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.545260:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75145216, 75149312) 0 0 00000080:00200000:2.0:1789760793.545263:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.545277:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18346, 18346] 00020000:00200000:2.0:1789760793.545284:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75145216 chunk: [75145216, 75149312) 75149312, 4096 00020000:00200000:2.0:1789760793.545288:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.545303:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18346, 18346] 00000008:00010000:2.0:1789760793.545317:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.545322:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.545330:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75145216, 75149312) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.545341:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75145216=0x47aa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.545394:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.545409:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75149312 00000020:00010000:2.0:1789760793.545416:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.545419:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.545422:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.545425:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.545428:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.545434:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.545439:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.545456:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760793.545551:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 75145216<>75149312 00000100:00100000:3.0:1789760793.546064:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760793.546092:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565579520:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.557309:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985654 gen 3 00000008:00010000:3.0:1789760793.557365:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760793.557374:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.557384:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.557415:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760793.557447:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565579520:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.557850:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18346, 18346] 00000080:00200000:2.0:1789760793.557873:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.557879:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.557893:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75149312 i_blocks 146800 00000020:00200000:2.0:1789760793.557898:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.558217:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.558246:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75149312, count: 4096 00000080:00200000:2.0:1789760793.558252:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75149312, bytes: 4096 00000080:00200000:2.0:1789760793.558264:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.558271:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.558273:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75149312, 75153408) 0 0 00000080:00200000:2.0:1789760793.558276:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.558290:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18347, 18347] 00020000:00200000:2.0:1789760793.558297:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75149312 chunk: [75149312, 75153408) 75153408, 4096 00020000:00200000:2.0:1789760793.558301:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.558319:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18347, 18347] 00000008:00010000:2.0:1789760793.558335:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.558341:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.558352:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75149312, 75153408) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.558364:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75149312=0x47ab000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.558496:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.558499:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75153408 00000020:00010000:2.0:1789760793.558506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.558510:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.558512:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.558515:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.558518:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.558524:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.558530:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.558548:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760793.559928:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 75149312<>75153408 00000100:00100000:3.0:1789760793.560001:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565579648:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.570494:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985655 gen 3 00000008:00010000:3.0:1789760793.570694:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:3.0:1789760793.570703:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.570715:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.570804:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:3.0:1789760793.570837:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565579648:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.571019:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18347, 18347] 00000080:00200000:1.0:1789760793.571091:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.571098:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.571114:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75153408 i_blocks 146808 00000020:00200000:1.0:1789760793.571118:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.571129:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.571166:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75153408, count: 4096 00000080:00200000:1.0:1789760793.571173:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75153408, bytes: 4096 00000080:00200000:1.0:1789760793.571183:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.571191:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.571193:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75153408, 75157504) 0 0 00000080:00200000:1.0:1789760793.571198:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.571215:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18348, 18348] 00020000:00200000:1.0:1789760793.571223:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75153408 chunk: [75153408, 75157504) 75157504, 4096 00020000:00200000:1.0:1789760793.571228:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.571247:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18348, 18348] 00000008:00010000:1.0:1789760793.571264:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.571343:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.571433:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75153408, 75157504) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.571560:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75153408=0x47ac000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.571627:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.571630:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75157504 00000020:00010000:1.0:1789760793.571638:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.571671:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.571675:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.571678:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.571681:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.571690:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.571697:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.571716:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.571844:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75153408<>75157504 00000100:00100000:0.0:1789760793.572126:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565579776:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.581326:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985656 gen 3 00000008:00010000:1.0:1789760793.581388:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.581406:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.581418:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.581442:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760793.581473:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565579776:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.581675:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18348, 18348] 00000080:00200000:3.0:1789760793.581693:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.581699:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760793.581724:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75157504 i_blocks 146816 00000020:00200000:3.0:1789760793.581729:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760793.581739:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760793.581773:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75157504, count: 4096 00000080:00200000:3.0:1789760793.581780:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75157504, bytes: 4096 00000080:00200000:3.0:1789760793.581790:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760793.581799:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760793.581801:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75157504, 75161600) 0 0 00000080:00200000:3.0:1789760793.581806:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760793.581823:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18349, 18349] 00020000:00200000:3.0:1789760793.581831:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75157504 chunk: [75157504, 75161600) 75161600, 4096 00020000:00200000:3.0:1789760793.581836:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760793.581856:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18349, 18349] 00000008:00010000:3.0:1789760793.581872:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760793.581878:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760793.581891:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75157504, 75161600) append 0 append lockpos 0 00000080:00200000:3.0:1789760793.581907:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75157504=0x47ad000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760793.581977:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760793.581980:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75161600 00000020:00010000:3.0:1789760793.581989:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:3.0:1789760793.581992:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.581995:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.581999:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760793.582002:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760793.582010:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760793.582017:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760793.582034:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760793.582164:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 75157504<>75161600 00000100:00100000:3.0:1789760793.582217:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760793.582234:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565579904:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.594997:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985657 gen 3 00000008:00010000:3.0:1789760793.595061:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:3.0:1789760793.595069:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.595081:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.595104:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:3.0:1789760793.595134:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565579904:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.595478:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18349, 18349] 00000080:00200000:2.0:1789760793.595498:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.595503:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.595516:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75161600 i_blocks 146824 00000020:00200000:2.0:1789760793.595522:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.595534:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.595562:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75161600, count: 4096 00000080:00200000:2.0:1789760793.595570:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75161600, bytes: 4096 00000080:00200000:2.0:1789760793.595582:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.595590:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.595592:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75161600, 75165696) 0 0 00000080:00200000:2.0:1789760793.595596:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.595612:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18350, 18350] 00020000:00200000:2.0:1789760793.595619:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75161600 chunk: [75161600, 75165696) 75165696, 4096 00020000:00200000:2.0:1789760793.595624:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.595641:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18350, 18350] 00000008:00010000:2.0:1789760793.595715:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.595720:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.595730:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75161600, 75165696) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.595750:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75161600=0x47ae000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.595803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.595806:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75165696 00000020:00010000:2.0:1789760793.595813:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.595816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.595818:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.595821:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.595824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.595830:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.596561:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.596587:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760793.596721:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 75161600<>75165696 00000100:00100000:2.0:1789760793.596776:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760793.596797:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565580032:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.608694:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985658 gen 3 00000008:00010000:2.0:1789760793.608735:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760793.608804:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.608813:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.608840:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760793.608866:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565580032:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.608996:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18350, 18350] 00000080:00200000:1.0:1789760793.609009:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.609014:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.609025:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75165696 i_blocks 146832 00000020:00200000:1.0:1789760793.609029:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.609036:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.609062:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75165696, count: 4096 00000080:00200000:1.0:1789760793.609068:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75165696, bytes: 4096 00000080:00200000:1.0:1789760793.609076:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.609083:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.609085:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75165696, 75169792) 0 0 00000080:00200000:1.0:1789760793.609089:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.609101:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18351, 18351] 00020000:00200000:1.0:1789760793.609108:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75165696 chunk: [75165696, 75169792) 75169792, 4096 00020000:00200000:1.0:1789760793.609112:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.609128:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18351, 18351] 00000008:00010000:1.0:1789760793.609142:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.609146:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.609156:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75165696, 75169792) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.609166:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75165696=0x47af000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.609209:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.609212:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75169792 00000020:00010000:1.0:1789760793.609219:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.609292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.609297:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.609301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.609363:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.609372:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.609378:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.609393:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.610013:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75165696<>75169792 00000100:00100000:0.0:1789760793.610081:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565580160:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.624507:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985659 gen 3 00000008:00010000:1.0:1789760793.624543:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.624550:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.624559:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.624580:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760793.624607:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565580160:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.624837:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18351, 18351] 00000080:00200000:2.0:1789760793.624851:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.624856:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.624867:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75169792 i_blocks 146840 00000020:00200000:2.0:1789760793.624870:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.624878:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.624904:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75169792, count: 4096 00000080:00200000:2.0:1789760793.624911:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75169792, bytes: 4096 00000080:00200000:2.0:1789760793.624919:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.624926:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.624928:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75169792, 75173888) 0 0 00000080:00200000:2.0:1789760793.624931:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.624944:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18352, 18352] 00020000:00200000:2.0:1789760793.624952:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75169792 chunk: [75169792, 75173888) 75173888, 4096 00020000:00200000:2.0:1789760793.624956:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.624972:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18352, 18352] 00000008:00010000:2.0:1789760793.624987:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.624992:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.625000:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75169792, 75173888) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.625012:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75169792=0x47b0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.625057:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.625060:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75173888 00000020:00010000:2.0:1789760793.625066:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.625069:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.625072:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.625075:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.625078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.625084:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.625089:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.625102:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760793.625320:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 75169792<>75173888 00000100:00100000:3.0:1789760793.625386:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565580288:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.641256:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985660 gen 3 00000008:00010000:2.0:1789760793.641533:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760793.641542:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.641553:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.641671:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760793.641705:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565580288:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.641903:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18352, 18352] 00000080:00200000:1.0:1789760793.641916:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.641921:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.641945:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75173888 i_blocks 146848 00000020:00200000:1.0:1789760793.641950:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.641959:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.641988:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75173888, count: 4096 00000080:00200000:1.0:1789760793.641994:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75173888, bytes: 4096 00000080:00200000:1.0:1789760793.642005:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.642013:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.642015:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75173888, 75177984) 0 0 00000080:00200000:1.0:1789760793.642019:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.642030:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18353, 18353] 00020000:00200000:1.0:1789760793.642037:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75173888 chunk: [75173888, 75177984) 75177984, 4096 00020000:00200000:1.0:1789760793.642041:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.642055:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18353, 18353] 00000008:00010000:1.0:1789760793.642769:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.642774:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.642784:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75173888, 75177984) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.642796:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75173888=0x47b1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.642844:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.642846:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75177984 00000020:00010000:1.0:1789760793.642854:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.642857:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.642860:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.642863:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.642865:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.642873:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.642879:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.642897:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.643011:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75173888<>75177984 00000100:00100000:0.0:1789760793.643097:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565580416:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.656008:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985661 gen 3 00000008:00010000:1.0:1789760793.656060:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.656068:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.656079:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.656229:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760793.656261:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565580416:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.658800:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18353, 18353] 00000080:00200000:2.0:1789760793.658827:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.658834:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.658853:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75177984 i_blocks 146856 00000020:00200000:2.0:1789760793.658858:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.658874:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.658911:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75177984, count: 4096 00000080:00200000:2.0:1789760793.658919:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75177984, bytes: 4096 00000080:00200000:2.0:1789760793.658933:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.658942:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.658945:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75177984, 75182080) 0 0 00000080:00200000:2.0:1789760793.658951:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.658970:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18354, 18354] 00020000:00200000:2.0:1789760793.658984:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75177984 chunk: [75177984, 75182080) 75182080, 4096 00020000:00200000:2.0:1789760793.658989:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.659013:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18354, 18354] 00000008:00010000:2.0:1789760793.659032:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.659038:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.659051:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75177984, 75182080) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.659067:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75177984=0x47b2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.659137:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.659140:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75182080 00000020:00010000:2.0:1789760793.659149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.659163:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.659167:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.659170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.659173:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.659182:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.659190:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.659219:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760793.659363:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 75177984<>75182080 00000100:00100000:2.0:1789760793.659429:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760793.659454:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565580544:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.672613:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985662 gen 3 00000008:00010000:2.0:1789760793.672708:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760793.672717:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.672789:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.672876:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760793.672907:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565580544:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.675725:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18354, 18354] 00000080:00200000:1.0:1789760793.677172:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.677185:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.677228:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75182080 i_blocks 146864 00000020:00200000:1.0:1789760793.677235:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.677252:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.677288:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75182080, count: 4096 00000080:00200000:1.0:1789760793.677297:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75182080, bytes: 4096 00000080:00200000:1.0:1789760793.677385:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.677407:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.677409:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75182080, 75186176) 0 0 00000080:00200000:1.0:1789760793.677414:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.677433:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18355, 18355] 00020000:00200000:1.0:1789760793.677442:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75182080 chunk: [75182080, 75186176) 75186176, 4096 00020000:00200000:1.0:1789760793.677447:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.677471:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18355, 18355] 00000008:00010000:1.0:1789760793.677503:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.677509:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.677521:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75182080, 75186176) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.677535:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75182080=0x47b3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.677607:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.677610:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75186176 00000020:00010000:1.0:1789760793.677620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.677623:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.677626:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.677630:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.677632:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.677641:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.677670:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.677691:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.677828:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75182080<>75186176 00000100:00100000:0.0:1789760793.677933:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565580672:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.692461:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985663 gen 3 00000008:00010000:1.0:1789760793.692513:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.692522:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.692533:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.692556:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760793.692587:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565580672:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.692827:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18355, 18355] 00000080:00200000:2.0:1789760793.692845:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.692851:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.692867:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75186176 i_blocks 146872 00000020:00200000:2.0:1789760793.692871:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.692882:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.692930:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75186176, count: 4096 00000080:00200000:2.0:1789760793.692938:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75186176, bytes: 4096 00000080:00200000:2.0:1789760793.692951:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.692961:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.692963:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75186176, 75190272) 0 0 00000080:00200000:2.0:1789760793.692968:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.692988:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18356, 18356] 00020000:00200000:2.0:1789760793.692997:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75186176 chunk: [75186176, 75190272) 75190272, 4096 00020000:00200000:2.0:1789760793.693002:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.693021:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18356, 18356] 00000008:00010000:2.0:1789760793.693039:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.693045:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.693057:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75186176, 75190272) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.693070:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75186176=0x47b4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.693123:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.693125:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75190272 00000020:00010000:2.0:1789760793.693134:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.693137:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.693139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.693143:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.693146:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.693152:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.693158:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.693173:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760793.693373:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 75186176<>75190272 00000100:00100000:3.0:1789760793.693509:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565580800:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.705637:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985664 gen 3 00000008:00010000:3.0:1789760793.705703:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760793.705711:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.705721:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.705753:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760793.705784:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565580800:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.706285:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18356, 18356] 00000080:00200000:2.0:1789760793.706303:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.706309:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.706324:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75190272 i_blocks 146880 00000020:00200000:2.0:1789760793.706328:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.706339:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.706373:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75190272, count: 4096 00000080:00200000:2.0:1789760793.706380:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75190272, bytes: 4096 00000080:00200000:2.0:1789760793.706391:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.706412:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.706414:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75190272, 75194368) 0 0 00000080:00200000:2.0:1789760793.706419:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.706444:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18357, 18357] 00020000:00200000:2.0:1789760793.706453:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75190272 chunk: [75190272, 75194368) 75194368, 4096 00020000:00200000:2.0:1789760793.706458:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.706477:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18357, 18357] 00000008:00010000:2.0:1789760793.706496:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.706502:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.706513:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75190272, 75194368) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.706528:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75190272=0x47b5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.706585:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.706588:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75194368 00000020:00010000:2.0:1789760793.706597:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.706601:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.706604:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.706608:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.706611:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.706618:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.706625:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.706665:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760793.706902:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 75190272<>75194368 00000100:00100000:3.0:1789760793.707128:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565580928:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.719219:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985665 gen 3 00000008:00010000:3.0:1789760793.719269:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760793.719277:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.719288:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.719312:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760793.719343:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565580928:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.719634:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18357, 18357] 00000080:00200000:2.0:1789760793.719686:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.719693:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.719709:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75194368 i_blocks 146888 00000020:00200000:2.0:1789760793.719713:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.720549:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.720585:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75194368, count: 4096 00000080:00200000:2.0:1789760793.720593:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75194368, bytes: 4096 00000080:00200000:2.0:1789760793.720607:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.720616:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.720618:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75194368, 75198464) 0 0 00000080:00200000:2.0:1789760793.720623:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.720640:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18358, 18358] 00020000:00200000:2.0:1789760793.720679:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75194368 chunk: [75194368, 75198464) 75198464, 4096 00020000:00200000:2.0:1789760793.720684:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.720705:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18358, 18358] 00000008:00010000:2.0:1789760793.720734:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.720748:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.720761:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75194368, 75198464) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.720776:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75194368=0x47b6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.720841:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.720845:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75198464 00000020:00010000:2.0:1789760793.720853:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.720857:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.720860:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.720863:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.720866:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.720875:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.720881:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.720904:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760793.721265:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 75194368<>75198464 00000100:00100000:3.0:1789760793.721697:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565581056:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.735119:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985666 gen 3 00000008:00010000:3.0:1789760793.735170:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:3.0:1789760793.735178:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.735189:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.735250:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000080:00200000:1.0:1789760793.735282:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18358, 18358] 00000100:00100000:3.0:1789760793.735286:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565581056:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.735304:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.735311:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.735328:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75198464 i_blocks 146896 00000020:00200000:1.0:1789760793.735332:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.735345:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.735379:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75198464, count: 4096 00000080:00200000:1.0:1789760793.735442:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75198464, bytes: 4096 00000080:00200000:1.0:1789760793.735454:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.735463:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.735466:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75198464, 75202560) 0 0 00000080:00200000:1.0:1789760793.735471:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.735490:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18359, 18359] 00020000:00200000:1.0:1789760793.735499:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75198464 chunk: [75198464, 75202560) 75202560, 4096 00020000:00200000:1.0:1789760793.735504:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.735525:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18359, 18359] 00000008:00010000:1.0:1789760793.735543:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.735549:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.735561:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75198464, 75202560) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.735575:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75198464=0x47b7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.735737:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.735741:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75202560 00000020:00010000:1.0:1789760793.735750:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.735754:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.735757:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.735760:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.735763:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.735770:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.735778:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.735796:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.735920:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75198464<>75202560 00000100:00100000:1.0:1789760793.735965:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760793.735982:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565581184:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.745269:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985667 gen 3 00000008:00010000:1.0:1789760793.745323:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.745340:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.745352:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.745371:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760793.745409:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565581184:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760793.745583:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18359, 18359] 00000080:00200000:0.0:1789760793.745598:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760793.745604:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760793.745619:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75202560 i_blocks 146904 00000020:00200000:0.0:1789760793.745622:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760793.745632:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760793.745709:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75202560, count: 4096 00000080:00200000:0.0:1789760793.745717:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75202560, bytes: 4096 00000080:00200000:0.0:1789760793.745729:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760793.745748:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760793.745750:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75202560, 75206656) 0 0 00000080:00200000:0.0:1789760793.745755:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760793.745772:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18360, 18360] 00020000:00200000:0.0:1789760793.745781:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75202560 chunk: [75202560, 75206656) 75206656, 4096 00020000:00200000:0.0:1789760793.745785:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760793.745806:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18360, 18360] 00000008:00010000:0.0:1789760793.745821:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760793.745827:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760793.745844:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75202560, 75206656) append 0 append lockpos 0 00000080:00200000:0.0:1789760793.745859:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75202560=0x47b8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760793.745991:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760793.745994:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75206656 00000020:00010000:0.0:1789760793.746004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:0.0:1789760793.746007:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760793.746011:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760793.746014:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760793.746017:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760793.746024:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760793.746031:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760793.746048:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760793.746167:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 75202560<>75206656 00000100:00100000:0.0:1789760793.746199:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760793.746214:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565581312:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.756307:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985668 gen 3 00000008:00010000:1.0:1789760793.756481:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760793.756539:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.756551:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.756608:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760793.756637:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565581312:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760793.756809:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18360, 18360] 00000080:00200000:0.0:1789760793.756824:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760793.756830:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760793.756848:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75206656 i_blocks 146912 00000020:00200000:0.0:1789760793.756852:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760793.756865:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760793.756972:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75206656, count: 4096 00000080:00200000:0.0:1789760793.756980:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75206656, bytes: 4096 00000080:00200000:0.0:1789760793.756990:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760793.756999:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760793.757001:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75206656, 75210752) 0 0 00000080:00200000:0.0:1789760793.757006:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760793.757024:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18361, 18361] 00020000:00200000:0.0:1789760793.757036:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75206656 chunk: [75206656, 75210752) 75210752, 4096 00020000:00200000:0.0:1789760793.757040:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760793.757061:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18361, 18361] 00000008:00010000:0.0:1789760793.757080:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760793.757087:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760793.757098:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75206656, 75210752) append 0 append lockpos 0 00000080:00200000:0.0:1789760793.757111:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75206656=0x47b9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760793.757171:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760793.757174:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75210752 00000020:00010000:0.0:1789760793.757184:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:0.0:1789760793.757187:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760793.757191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760793.757194:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760793.757197:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760793.757203:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760793.757210:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760793.757228:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760793.758050:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75206656<>75210752 00000100:00100000:1.0:1789760793.758123:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565581440:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.768857:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985669 gen 3 00000008:00010000:1.0:1789760793.768903:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760793.768911:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.768922:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.769056:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760793.769091:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565581440:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.770699:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18361, 18361] 00000080:00200000:3.0:1789760793.770724:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.770732:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.771000:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75210752 i_blocks 146920 00000020:00200000:1.0:1789760793.771007:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.771023:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.771060:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75210752, count: 4096 00000080:00200000:1.0:1789760793.771068:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75210752, bytes: 4096 00000080:00200000:1.0:1789760793.771083:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.771093:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.771096:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75210752, 75214848) 0 0 00000080:00200000:1.0:1789760793.771101:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.771120:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18362, 18362] 00020000:00200000:1.0:1789760793.771131:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75210752 chunk: [75210752, 75214848) 75214848, 4096 00020000:00200000:1.0:1789760793.771137:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.771159:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18362, 18362] 00000008:00010000:1.0:1789760793.771177:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.771183:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.771194:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75210752, 75214848) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.771208:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75210752=0x47ba000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.771270:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.771274:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75214848 00000020:00010000:1.0:1789760793.771282:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.771285:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.771288:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.771291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.771294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.771303:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.771310:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.771328:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.771468:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75210752<>75214848 00000100:00100000:1.0:1789760793.771523:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760793.771548:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565581568:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.783619:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985670 gen 3 00000008:00010000:1.0:1789760793.783841:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.783850:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.783862:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.783927:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760793.783956:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565581568:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.784079:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18362, 18362] 00000080:00200000:2.0:1789760793.784096:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.784102:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.784119:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75214848 i_blocks 146928 00000020:00200000:2.0:1789760793.784122:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.784132:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.784165:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75214848, count: 4096 00000080:00200000:2.0:1789760793.784173:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75214848, bytes: 4096 00000080:00200000:2.0:1789760793.784185:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.784194:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.784197:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75214848, 75218944) 0 0 00000080:00200000:2.0:1789760793.784202:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.784220:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18363, 18363] 00020000:00200000:2.0:1789760793.784229:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75214848 chunk: [75214848, 75218944) 75218944, 4096 00020000:00200000:2.0:1789760793.784234:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.784255:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18363, 18363] 00000008:00010000:2.0:1789760793.784274:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.784279:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.784291:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75214848, 75218944) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.784305:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75214848=0x47bb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.784361:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.784364:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75218944 00000020:00010000:2.0:1789760793.784372:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.784376:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.784379:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.784383:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.784386:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.784810:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.784821:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.784840:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354ff00 00000008:00100000:2.0:1789760793.785258:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 75214848<>75218944 00000100:00100000:3.0:1789760793.785444:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760793.785462:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565581696:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.793921:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985671 gen 3 00000008:00010000:3.0:1789760793.793971:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354ff00 00000020:00200000:3.0:1789760793.793979:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.793991:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.794102:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354ff00 00000100:00100000:3.0:1789760793.794136:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565581696:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.794822:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18363, 18363] 00000080:00200000:2.0:1789760793.794840:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.794847:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.794863:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75218944 i_blocks 146936 00000020:00200000:2.0:1789760793.794867:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.794878:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.794912:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75218944, count: 4096 00000080:00200000:2.0:1789760793.794919:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75218944, bytes: 4096 00000080:00200000:2.0:1789760793.794931:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.794939:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.794941:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75218944, 75223040) 0 0 00000080:00200000:2.0:1789760793.794946:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.794963:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18364, 18364] 00020000:00200000:2.0:1789760793.794972:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75218944 chunk: [75218944, 75223040) 75223040, 4096 00020000:00200000:2.0:1789760793.794977:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.794996:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18364, 18364] 00000008:00010000:2.0:1789760793.795017:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.795023:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.795034:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75218944, 75223040) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.795048:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75218944=0x47bc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.795111:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.795114:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75223040 00000020:00010000:2.0:1789760793.795123:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.795126:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.795129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.795133:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.795135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.795143:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.795266:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.795290:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f180 00000008:00100000:2.0:1789760793.795425:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 75218944<>75223040 00000100:00100000:3.0:1789760793.795903:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760793.795986:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565581824:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.808334:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985672 gen 3 00000008:00010000:3.0:1789760793.808389:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f180 00000020:00200000:3.0:1789760793.808411:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.808424:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.808451:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f180 00000100:00100000:3.0:1789760793.808486:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565581824:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.808527:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18364, 18364] 00000080:00200000:2.0:1789760793.808544:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.808549:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.808572:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75223040 i_blocks 146944 00000020:00200000:2.0:1789760793.808576:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.808587:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.808621:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75223040, count: 4096 00000080:00200000:2.0:1789760793.808629:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75223040, bytes: 4096 00000080:00200000:2.0:1789760793.808639:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.808674:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.808676:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75223040, 75227136) 0 0 00000080:00200000:2.0:1789760793.808680:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.808697:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18365, 18365] 00020000:00200000:2.0:1789760793.808716:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75223040 chunk: [75223040, 75227136) 75227136, 4096 00020000:00200000:2.0:1789760793.808721:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.808739:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18365, 18365] 00000008:00010000:2.0:1789760793.808956:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.808962:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.808979:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75223040, 75227136) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.808994:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75223040=0x47bd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.809047:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.809050:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75227136 00000020:00010000:2.0:1789760793.809058:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.809061:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.809065:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.809069:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.809071:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.809079:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.809086:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760793.809227:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f180 00000008:00100000:3.0:1789760793.809359:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 75223040<>75227136 00000100:00100000:3.0:1789760793.809433:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565581952:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.818725:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985673 gen 3 00000008:00010000:2.0:1789760793.818785:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f180 00000020:00200000:2.0:1789760793.818793:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.818806:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.818891:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f180 00000100:00100000:2.0:1789760793.818925:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565581952:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.819317:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18365, 18365] 00000080:00200000:1.0:1789760793.819859:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.819868:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.819884:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75227136 i_blocks 146952 00000020:00200000:1.0:1789760793.819888:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.819900:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.819934:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75227136, count: 4096 00000080:00200000:1.0:1789760793.819942:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75227136, bytes: 4096 00000080:00200000:1.0:1789760793.819954:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.819963:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.819965:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75227136, 75231232) 0 0 00000080:00200000:1.0:1789760793.819970:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.819987:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18366, 18366] 00020000:00200000:1.0:1789760793.819996:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75227136 chunk: [75227136, 75231232) 75231232, 4096 00020000:00200000:1.0:1789760793.820001:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.820021:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18366, 18366] 00000008:00010000:1.0:1789760793.820038:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.820045:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.820058:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75227136, 75231232) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.820071:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75227136=0x47be000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.820137:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.820140:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75231232 00000020:00010000:1.0:1789760793.820149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.820152:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.820156:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.820159:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.820162:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.820169:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.820175:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.820199:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.820348:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75227136<>75231232 00000100:00100000:0.0:1789760793.820434:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760793.820458:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565582080:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.830714:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985674 gen 3 00000008:00010000:1.0:1789760793.830813:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.830822:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.830832:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.830917:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000080:00200000:2.0:1789760793.830939:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18366, 18366] 00000100:00100000:1.0:1789760793.830948:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565582080:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.830956:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.830962:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.830976:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75231232 i_blocks 146960 00000020:00200000:2.0:1789760793.830979:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.830989:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.831024:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75231232, count: 4096 00000080:00200000:2.0:1789760793.831032:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75231232, bytes: 4096 00000080:00200000:2.0:1789760793.831043:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.831050:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.831053:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75231232, 75235328) 0 0 00000080:00200000:2.0:1789760793.831058:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.831075:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18367, 18367] 00020000:00200000:2.0:1789760793.831085:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75231232 chunk: [75231232, 75235328) 75235328, 4096 00020000:00200000:2.0:1789760793.831089:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.831108:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18367, 18367] 00000008:00010000:2.0:1789760793.831124:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.831129:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.831141:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75231232, 75235328) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.831154:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75231232=0x47bf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.831210:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.831213:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75235328 00000020:00010000:2.0:1789760793.831221:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.831224:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.831226:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.831229:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.831232:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.831238:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.831244:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.831258:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f180 00000008:00100000:2.0:1789760793.831894:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 75231232<>75235328 00000100:00100000:3.0:1789760793.831975:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565582208:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.841722:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985675 gen 3 00000008:00010000:2.0:1789760793.841786:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f180 00000020:00200000:2.0:1789760793.841795:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.841807:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.841837:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f180 00000100:00100000:2.0:1789760793.841869:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565582208:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.841915:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18367, 18367] 00000080:00200000:1.0:1789760793.841933:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.841940:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.841955:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75235328 i_blocks 146968 00000020:00200000:1.0:1789760793.841959:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.841970:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.842007:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75235328, count: 4096 00000080:00200000:1.0:1789760793.842016:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75235328, bytes: 4096 00000080:00200000:1.0:1789760793.842026:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.842035:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.842038:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75235328, 75239424) 0 0 00000080:00200000:1.0:1789760793.842042:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.842060:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18368, 18368] 00020000:00200000:1.0:1789760793.842069:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75235328 chunk: [75235328, 75239424) 75239424, 4096 00020000:00200000:1.0:1789760793.842074:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.842094:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18368, 18368] 00000008:00010000:1.0:1789760793.842112:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.842119:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.842131:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75235328, 75239424) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.842146:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75235328=0x47c0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.842205:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.842208:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75239424 00000020:00010000:1.0:1789760793.842218:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.842221:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.842224:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.842228:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.842231:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.842238:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.842245:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.842261:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.842405:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75235328<>75239424 00000100:00100000:0.0:1789760793.842472:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760793.842501:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565582336:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.851225:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985676 gen 3 00000008:00010000:1.0:1789760793.851278:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.851286:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.851296:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.851320:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760793.851350:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565582336:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.851940:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18368, 18368] 00000080:00200000:2.0:1789760793.851958:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.851965:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.851980:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75239424 i_blocks 146976 00000020:00200000:2.0:1789760793.851984:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.851994:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.852031:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75239424, count: 4096 00000080:00200000:2.0:1789760793.852039:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75239424, bytes: 4096 00000080:00200000:2.0:1789760793.852051:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.852059:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.852062:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75239424, 75243520) 0 0 00000080:00200000:2.0:1789760793.852067:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.852084:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18369, 18369] 00020000:00200000:2.0:1789760793.852093:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75239424 chunk: [75239424, 75243520) 75243520, 4096 00020000:00200000:2.0:1789760793.852098:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.852118:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18369, 18369] 00000008:00010000:2.0:1789760793.852136:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.852142:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.852155:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75239424, 75243520) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.852168:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75239424=0x47c1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.852228:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.852231:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75243520 00000020:00010000:2.0:1789760793.852240:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.852244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.852247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a3f8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.852251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d08a8 (ffff93f84418af38/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.852254:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418af38/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.852263:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.852270:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.852287:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f180 00000008:00100000:2.0:1789760793.853048:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 75239424<>75243520 00000100:00100000:2.0:1789760793.853116:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565582464:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.860165:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985677 gen 3 00000008:00010000:2.0:1789760793.860217:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f180 00000020:00200000:2.0:1789760793.860225:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.860236:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.860300:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f180 00000100:00100000:2.0:1789760793.860340:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565582464:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.860481:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18369, 18369] 00000080:00200000:3.0:1789760793.860543:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.860548:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760793.860562:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75243520 i_blocks 146984 00000020:00200000:3.0:1789760793.860565:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760793.860575:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760793.860638:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75243520, count: 4096 00000080:00200000:3.0:1789760793.860687:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75243520, bytes: 4096 00000080:00200000:3.0:1789760793.860698:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760793.860706:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760793.860708:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75243520, 75247616) 0 0 00000080:00200000:3.0:1789760793.860713:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760793.860729:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18370, 18370] 00020000:00200000:3.0:1789760793.860738:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75243520 chunk: [75243520, 75247616) 75247616, 4096 00020000:00200000:3.0:1789760793.860852:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760793.860874:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18370, 18370] 00000008:00010000:3.0:1789760793.860890:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760793.860896:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760793.860906:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75243520, 75247616) append 0 append lockpos 0 00000080:00200000:3.0:1789760793.860919:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75243520=0x47c2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760793.860978:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760793.860981:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75247616 00000020:00010000:3.0:1789760793.860989:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0038 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760793.860992:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0038 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.860996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.861000:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0038 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760793.861003:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760793.861011:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760793.861018:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760793.861039:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760793.861161:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 75243520<>75247616 00000100:00100000:2.0:1789760793.861228:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565582592:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.873414:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985678 gen 3 00000008:00010000:2.0:1789760793.873469:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760793.873477:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.873489:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.873581:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760793.873612:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565582592:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.873734:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18370, 18370] 00000080:00200000:3.0:1789760793.873752:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.873757:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.873863:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75247616 i_blocks 146992 00000020:00200000:2.0:1789760793.874216:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.874228:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.874264:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75247616, count: 4096 00000080:00200000:2.0:1789760793.874274:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75247616, bytes: 4096 00000080:00200000:2.0:1789760793.874289:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.874298:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.874301:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75247616, 75251712) 0 0 00000080:00200000:2.0:1789760793.874305:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.874322:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18371, 18371] 00020000:00200000:2.0:1789760793.874333:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75247616 chunk: [75247616, 75251712) 75251712, 4096 00020000:00200000:2.0:1789760793.874337:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.874359:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18371, 18371] 00000008:00010000:2.0:1789760793.874377:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.874383:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.874403:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75247616, 75251712) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.874418:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75247616=0x47c3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.874482:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.874486:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75251712 00000020:00010000:2.0:1789760793.874495:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d0038 (ffff93f84418a3f8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.874499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d0038 (ffff93f84418a3f8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.874502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.874506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d0038 (ffff93f84418a3f8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.874509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a3f8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.874517:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.874525:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.874576:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f180 00000008:00100000:2.0:1789760793.875070:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 75247616<>75251712 00000100:00100000:3.0:1789760793.875141:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565582720:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.890941:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985679 gen 3 00000008:00010000:2.0:1789760793.891849:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f180 00000020:00200000:2.0:1789760793.891860:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.891871:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.891900:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f180 00000100:00100000:2.0:1789760793.891948:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565582720:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760793.892129:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18371, 18371] 00000080:00200000:3.0:1789760793.892146:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760793.892153:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760793.892168:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75251712 i_blocks 147000 00000020:00200000:3.0:1789760793.892172:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760793.892182:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760793.892245:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75251712, count: 4096 00000080:00200000:3.0:1789760793.892254:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75251712, bytes: 4096 00000080:00200000:3.0:1789760793.892271:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760793.892280:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760793.892282:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75251712, 75255808) 0 0 00000080:00200000:3.0:1789760793.892287:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760793.892302:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18372, 18372] 00020000:00200000:3.0:1789760793.892310:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75251712 chunk: [75251712, 75255808) 75255808, 4096 00020000:00200000:3.0:1789760793.892315:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760793.892333:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18372, 18372] 00000008:00010000:3.0:1789760793.892348:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760793.892354:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760793.892365:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75251712, 75255808) append 0 append lockpos 0 00000080:00200000:3.0:1789760793.892378:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75251712=0x47c4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760793.892450:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760793.892453:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75255808 00000020:00010000:3.0:1789760793.892461:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760793.892465:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.892468:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760793.892472:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760793.892475:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760793.892483:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760793.892490:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760793.892507:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760793.892633:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 75251712<>75255808 00000100:00100000:2.0:1789760793.892750:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565582848:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.904327:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985680 gen 3 00000008:00010000:2.0:1789760793.904415:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760793.904424:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.904437:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.904517:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760793.904551:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565582848:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.907070:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18372, 18372] 00000080:00200000:1.0:1789760793.907246:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.907253:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.907270:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75255808 i_blocks 147008 00000020:00200000:1.0:1789760793.907276:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.907293:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.907338:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75255808, count: 4096 00000080:00200000:1.0:1789760793.907347:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75255808, bytes: 4096 00000080:00200000:1.0:1789760793.907361:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.907370:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.907373:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75255808, 75259904) 0 0 00000080:00200000:1.0:1789760793.907377:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.907407:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18373, 18373] 00020000:00200000:1.0:1789760793.907418:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75255808 chunk: [75255808, 75259904) 75259904, 4096 00020000:00200000:1.0:1789760793.907423:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.907447:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18373, 18373] 00000008:00010000:1.0:1789760793.907466:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.907472:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.907924:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75255808, 75259904) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.907942:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75255808=0x47c5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.908021:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.908024:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75259904 00000020:00010000:1.0:1789760793.908035:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.908039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.908042:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.908046:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.908049:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.908059:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.908067:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.908098:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.908238:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75255808<>75259904 00000100:00100000:1.0:1789760793.908298:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760793.908336:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565582976:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.921744:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985681 gen 3 00000008:00010000:1.0:1789760793.921851:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.921861:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.921872:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.921947:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760793.921978:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565582976:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760793.922100:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18373, 18373] 00000080:00200000:0.0:1789760793.922117:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760793.922123:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760793.922140:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75259904 i_blocks 147016 00000020:00200000:0.0:1789760793.922143:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760793.922152:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760793.922187:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75259904, count: 4096 00000080:00200000:0.0:1789760793.922196:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75259904, bytes: 4096 00000080:00200000:0.0:1789760793.922255:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760793.922265:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760793.922268:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75259904, 75264000) 0 0 00000080:00200000:0.0:1789760793.922273:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760793.922289:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18374, 18374] 00020000:00200000:0.0:1789760793.922298:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75259904 chunk: [75259904, 75264000) 75264000, 4096 00020000:00200000:0.0:1789760793.922303:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760793.922324:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18374, 18374] 00000008:00010000:0.0:1789760793.922412:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760793.922419:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760793.922431:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75259904, 75264000) append 0 append lockpos 0 00000080:00200000:0.0:1789760793.922445:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75259904=0x47c6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760793.922504:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760793.922508:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75264000 00000020:00010000:0.0:1789760793.922516:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760793.922520:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760793.922523:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760793.922526:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760793.922529:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760793.922535:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760793.922543:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760793.922562:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760793.924148:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 75259904<>75264000 00000100:00100000:1.0:1789760793.924251:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565583104:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.935060:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985682 gen 3 00000008:00010000:1.0:1789760793.935112:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760793.935121:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.935132:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.935167:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760793.935199:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565583104:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.936481:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18374, 18374] 00000080:00200000:2.0:1789760793.936505:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.936511:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.936528:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75264000 i_blocks 147024 00000020:00200000:2.0:1789760793.936533:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.936548:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.936583:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75264000, count: 4096 00000080:00200000:2.0:1789760793.936592:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75264000, bytes: 4096 00000080:00200000:2.0:1789760793.936605:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.936614:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.936617:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75264000, 75268096) 0 0 00000080:00200000:2.0:1789760793.936622:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.936639:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18375, 18375] 00020000:00200000:2.0:1789760793.936688:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75264000 chunk: [75264000, 75268096) 75268096, 4096 00020000:00200000:2.0:1789760793.936694:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.936717:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18375, 18375] 00000008:00010000:2.0:1789760793.936735:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.936741:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.936753:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75264000, 75268096) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.936767:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75264000=0x47c7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.936832:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.936835:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75268096 00000020:00010000:2.0:1789760793.936844:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.936847:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.936850:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.936854:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.936857:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.936865:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.936875:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.936898:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f180 00000008:00100000:2.0:1789760793.937043:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 75264000<>75268096 00000100:00100000:2.0:1789760793.937102:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760793.937126:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565583232:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760793.948606:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985683 gen 3 00000008:00010000:2.0:1789760793.948693:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f180 00000020:00200000:2.0:1789760793.948701:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760793.948712:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760793.948741:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f180 00000100:00100000:2.0:1789760793.948772:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565583232:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760793.948887:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18375, 18375] 00000080:00200000:1.0:1789760793.948903:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760793.948909:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760793.948935:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75268096 i_blocks 147032 00000020:00200000:1.0:1789760793.948938:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760793.948948:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760793.948980:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75268096, count: 4096 00000080:00200000:1.0:1789760793.948987:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75268096, bytes: 4096 00000080:00200000:1.0:1789760793.948997:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760793.949005:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760793.949007:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75268096, 75272192) 0 0 00000080:00200000:1.0:1789760793.949011:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760793.949027:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18376, 18376] 00020000:00200000:1.0:1789760793.949035:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75268096 chunk: [75268096, 75272192) 75272192, 4096 00020000:00200000:1.0:1789760793.949040:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760793.949058:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18376, 18376] 00000008:00010000:1.0:1789760793.949074:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760793.949080:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760793.949090:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75268096, 75272192) append 0 append lockpos 0 00000080:00200000:1.0:1789760793.949102:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75268096=0x47c8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760793.949155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760793.949157:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75272192 00000020:00010000:1.0:1789760793.949164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760793.949167:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.949170:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760793.949173:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760793.949176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760793.949182:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760793.949189:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760793.949203:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760793.949325:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75268096<>75272192 00000100:00100000:1.0:1789760793.949371:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760793.949387:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565583360:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.961082:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985684 gen 3 00000008:00010000:1.0:1789760793.961135:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760793.961144:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.961155:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760793.961178:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760793.961218:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565583360:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760793.961330:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18376, 18376] 00000080:00200000:0.0:1789760793.961346:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760793.961352:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760793.961368:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75272192 i_blocks 147040 00000020:00200000:0.0:1789760793.961372:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760793.961382:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760793.961423:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75272192, count: 4096 00000080:00200000:0.0:1789760793.961431:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75272192, bytes: 4096 00000080:00200000:0.0:1789760793.961442:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760793.961531:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760793.961534:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75272192, 75276288) 0 0 00000080:00200000:0.0:1789760793.961540:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760793.961558:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18377, 18377] 00020000:00200000:0.0:1789760793.961567:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75272192 chunk: [75272192, 75276288) 75276288, 4096 00020000:00200000:0.0:1789760793.961572:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760793.961592:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18377, 18377] 00000008:00010000:0.0:1789760793.961611:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760793.961618:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760793.961630:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75272192, 75276288) append 0 append lockpos 0 00000080:00200000:0.0:1789760793.961690:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75272192=0x47c9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760793.961745:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760793.961748:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75276288 00000020:00010000:0.0:1789760793.961756:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760793.961759:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760793.961762:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760793.961765:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760793.961769:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760793.961776:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760793.961782:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760793.961798:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dcc0 00000008:00100000:0.0:1789760793.962121:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42a00 - object 0x240000400:5 offset 75272192<>75276288 00000100:00100000:0.0:1789760793.962162:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760793.962182:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565583488:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760793.972334:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985685 gen 3 00000008:00010000:1.0:1789760793.972387:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760793.972413:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760793.972425:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.973248:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18377, 18377] 00000080:00200000:2.0:1789760793.973269:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.973275:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.973293:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75276288 i_blocks 147048 00000020:00200000:2.0:1789760793.973298:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.973313:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.973365:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75276288, count: 4096 00000080:00200000:2.0:1789760793.973374:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75276288, bytes: 4096 00000080:00200000:2.0:1789760793.973388:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.973406:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.973408:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75276288, 75280384) 0 0 00000080:00200000:2.0:1789760793.973413:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.973430:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18378, 18378] 00020000:00200000:2.0:1789760793.973440:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75276288 chunk: [75276288, 75280384) 75280384, 4096 00020000:00200000:2.0:1789760793.973444:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:1.0:1789760793.973734:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760793.973769:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565583488:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.973896:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18378, 18378] 00000008:00010000:2.0:1789760793.973917:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.973923:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.973937:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75276288, 75280384) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.973952:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75276288=0x47ca000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.974013:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.974017:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75280384 00000020:00010000:2.0:1789760793.974025:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.974029:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.974032:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.974035:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.974038:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.974046:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.974052:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.974069:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f180 00000008:00100000:2.0:1789760793.974217:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 75276288<>75280384 00000100:00100000:3.0:1789760793.974992:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565583616:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760793.988412:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985686 gen 3 00000008:00010000:3.0:1789760793.988538:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f180 00000020:00200000:3.0:1789760793.988548:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760793.988558:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760793.988580:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f180 00000100:00100000:3.0:1789760793.988613:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565583616:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760793.988816:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18378, 18378] 00000080:00200000:2.0:1789760793.988832:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760793.988838:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760793.988852:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75280384 i_blocks 147056 00000020:00200000:2.0:1789760793.988855:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760793.988870:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760793.988903:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75280384, count: 4096 00000080:00200000:2.0:1789760793.988909:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75280384, bytes: 4096 00000080:00200000:2.0:1789760793.988919:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760793.988929:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760793.988931:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75280384, 75284480) 0 0 00000080:00200000:2.0:1789760793.988935:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760793.988951:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18379, 18379] 00020000:00200000:2.0:1789760793.988959:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75280384 chunk: [75280384, 75284480) 75284480, 4096 00020000:00200000:2.0:1789760793.988965:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760793.988984:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18379, 18379] 00000008:00010000:2.0:1789760793.989002:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760793.989008:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760793.989020:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75280384, 75284480) append 0 append lockpos 0 00000080:00200000:2.0:1789760793.989034:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75280384=0x47cb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760793.989094:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760793.989097:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75284480 00000020:00010000:2.0:1789760793.989105:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760793.989109:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.989112:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760793.989115:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760793.989118:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760793.989126:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760793.989133:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760793.989149:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760793.990997:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 75280384<>75284480 00000100:00100000:2.0:1789760793.991096:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565583744:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.004009:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985687 gen 3 00000008:00010000:2.0:1789760794.004148:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:2.0:1789760794.004157:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.004169:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.004195:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:2.0:1789760794.004226:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565583744:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.004412:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18379, 18379] 00000080:00200000:3.0:1789760794.004428:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.004434:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760794.004449:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75284480 i_blocks 147064 00000020:00200000:3.0:1789760794.004453:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760794.004464:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760794.004498:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75284480, count: 4096 00000080:00200000:3.0:1789760794.004506:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75284480, bytes: 4096 00000080:00200000:3.0:1789760794.004518:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760794.004527:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760794.004529:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75284480, 75288576) 0 0 00000080:00200000:3.0:1789760794.004534:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760794.004550:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18380, 18380] 00020000:00200000:3.0:1789760794.004560:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75284480 chunk: [75284480, 75288576) 75288576, 4096 00020000:00200000:3.0:1789760794.004564:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760794.004675:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18380, 18380] 00000008:00010000:3.0:1789760794.004696:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760794.004702:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760794.004778:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75284480, 75288576) append 0 append lockpos 0 00000080:00200000:3.0:1789760794.004795:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75284480=0x47cc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760794.004852:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760794.004855:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75288576 00000020:00010000:3.0:1789760794.004863:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760794.004867:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.004870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.004873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760794.004876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760794.004885:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760794.004892:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760794.004908:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760794.005031:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 75284480<>75288576 00000100:00100000:2.0:1789760794.005282:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565583872:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.016530:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985688 gen 3 00000008:00010000:2.0:1789760794.016579:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760794.016588:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.016599:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.016624:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760794.016694:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565583872:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.016840:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18380, 18380] 00000080:00200000:3.0:1789760794.016856:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.016862:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760794.016878:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75288576 i_blocks 147072 00000020:00200000:3.0:1789760794.016882:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760794.016892:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760794.016927:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75288576, count: 4096 00000080:00200000:3.0:1789760794.016934:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75288576, bytes: 4096 00000080:00200000:3.0:1789760794.016945:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760794.016955:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760794.016957:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75288576, 75292672) 0 0 00000080:00200000:3.0:1789760794.016962:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760794.016980:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18381, 18381] 00020000:00200000:3.0:1789760794.016989:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75288576 chunk: [75288576, 75292672) 75292672, 4096 00020000:00200000:3.0:1789760794.016994:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760794.017015:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18381, 18381] 00000008:00010000:3.0:1789760794.017035:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760794.017041:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760794.017052:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75288576, 75292672) append 0 append lockpos 0 00000080:00200000:3.0:1789760794.017066:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75288576=0x47cd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760794.017123:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760794.017126:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75292672 00000020:00010000:3.0:1789760794.017134:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760794.017138:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.017141:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.017144:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760794.017147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760794.017155:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760794.017162:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760794.017180:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760794.017473:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 75288576<>75292672 00000100:00100000:2.0:1789760794.017535:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565584000:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.026728:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985689 gen 3 00000008:00010000:2.0:1789760794.026790:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760794.026800:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.026812:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.026876:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:2.0:1789760794.026907:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565584000:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.027902:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18381, 18381] 00000080:00200000:3.0:1789760794.027919:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.027924:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760794.027938:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75292672 i_blocks 147080 00000020:00200000:3.0:1789760794.027941:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760794.027951:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760794.027983:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75292672, count: 4096 00000080:00200000:3.0:1789760794.027989:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75292672, bytes: 4096 00000080:00200000:3.0:1789760794.028000:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760794.028007:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760794.028009:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75292672, 75296768) 0 0 00000080:00200000:3.0:1789760794.028013:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760794.028027:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18382, 18382] 00020000:00200000:3.0:1789760794.028035:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75292672 chunk: [75292672, 75296768) 75296768, 4096 00020000:00200000:3.0:1789760794.028040:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760794.028061:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18382, 18382] 00000008:00010000:3.0:1789760794.028078:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760794.028084:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760794.028097:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75292672, 75296768) append 0 append lockpos 0 00000080:00200000:3.0:1789760794.028111:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75292672=0x47ce000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760794.028176:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760794.028179:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75296768 00000020:00010000:3.0:1789760794.028188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760794.028191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.028194:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.028198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760794.028201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760794.028211:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760794.028218:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760794.028247:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760794.028501:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 75292672<>75296768 00000100:00100000:2.0:1789760794.029210:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565584128:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.037908:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985690 gen 3 00000008:00010000:2.0:1789760794.037962:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760794.037971:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.037985:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.038007:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760794.038037:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565584128:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.038158:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18382, 18382] 00000080:00200000:3.0:1789760794.038176:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.038182:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760794.038197:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75296768 i_blocks 147088 00000020:00200000:3.0:1789760794.038200:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760794.038210:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760794.038244:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75296768, count: 4096 00000080:00200000:3.0:1789760794.038252:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75296768, bytes: 4096 00000080:00200000:3.0:1789760794.038263:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760794.038272:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760794.038275:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75296768, 75300864) 0 0 00000080:00200000:3.0:1789760794.038280:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760794.038296:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18383, 18383] 00020000:00200000:3.0:1789760794.038305:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75296768 chunk: [75296768, 75300864) 75300864, 4096 00020000:00200000:3.0:1789760794.038309:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760794.038340:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18383, 18383] 00000008:00010000:3.0:1789760794.038357:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760794.038363:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760794.038374:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75296768, 75300864) append 0 append lockpos 0 00000080:00200000:3.0:1789760794.038388:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75296768=0x47cf000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760794.038468:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760794.038471:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75300864 00000020:00010000:3.0:1789760794.038479:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760794.038483:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.038487:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.038490:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760794.038493:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760794.038501:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760794.038507:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760794.038522:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760794.038695:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 75296768<>75300864 00000100:00100000:2.0:1789760794.038941:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760794.038959:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565584256:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.050210:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985691 gen 3 00000008:00010000:2.0:1789760794.050260:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760794.050269:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.050280:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.050304:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760794.050333:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565584256:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.050447:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18383, 18383] 00000080:00200000:3.0:1789760794.050466:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.050472:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760794.050489:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75300864 i_blocks 147096 00000020:00200000:3.0:1789760794.050492:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760794.050508:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760794.050542:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75300864, count: 4096 00000080:00200000:3.0:1789760794.050549:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75300864, bytes: 4096 00000080:00200000:3.0:1789760794.050560:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760794.050568:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760794.050571:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75300864, 75304960) 0 0 00000080:00200000:3.0:1789760794.050575:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760794.050592:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18384, 18384] 00020000:00200000:3.0:1789760794.050600:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75300864 chunk: [75300864, 75304960) 75304960, 4096 00020000:00200000:3.0:1789760794.050605:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760794.050625:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18384, 18384] 00000008:00010000:3.0:1789760794.050915:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760794.050923:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760794.050937:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75300864, 75304960) append 0 append lockpos 0 00000080:00200000:3.0:1789760794.050950:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75300864=0x47d0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760794.051010:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760794.051013:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75304960 00000020:00010000:3.0:1789760794.051021:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760794.051025:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.051027:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.051031:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760794.051033:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760794.051042:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760794.051050:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760794.051069:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760794.051195:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66680 - object 0x240000400:5 offset 75300864<>75304960 00000100:00100000:2.0:1789760794.051310:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760794.051328:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565584384:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.068628:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985692 gen 3 00000008:00010000:2.0:1789760794.069103:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760794.069114:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.069128:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.069202:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000080:00200000:3.0:1789760794.069234:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18384, 18384] 00000100:00100000:2.0:1789760794.069236:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565584384:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.069259:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.069265:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760794.069281:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75304960 i_blocks 147104 00000020:00200000:3.0:1789760794.069284:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760794.069295:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760794.069405:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75304960, count: 4096 00000080:00200000:3.0:1789760794.069414:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75304960, bytes: 4096 00000080:00200000:3.0:1789760794.069430:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760794.069438:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760794.069440:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75304960, 75309056) 0 0 00000080:00200000:3.0:1789760794.069445:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760794.069464:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18385, 18385] 00020000:00200000:3.0:1789760794.069473:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75304960 chunk: [75304960, 75309056) 75309056, 4096 00020000:00200000:3.0:1789760794.069478:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760794.069498:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18385, 18385] 00000008:00010000:3.0:1789760794.069516:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760794.069522:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760794.069534:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75304960, 75309056) append 0 append lockpos 0 00000080:00200000:3.0:1789760794.069547:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75304960=0x47d1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760794.069607:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760794.069611:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75309056 00000020:00010000:3.0:1789760794.069619:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760794.069622:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.069626:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.069629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760794.069632:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760794.069640:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760794.070429:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760794.070463:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760794.070596:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67b80 - object 0x240000400:5 offset 75304960<>75309056 00000100:00100000:2.0:1789760794.071262:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760794.071291:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565584512:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.111906:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985693 gen 3 00000008:00010000:2.0:1789760794.111961:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760794.111969:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.111981:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.112006:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760794.112037:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565584512:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.112150:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18385, 18385] 00000080:00200000:3.0:1789760794.112167:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.112173:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760794.112188:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75309056 i_blocks 147112 00000020:00200000:3.0:1789760794.112192:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760794.112203:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760794.112248:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75309056, count: 4096 00000080:00200000:3.0:1789760794.112255:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75309056, bytes: 4096 00000080:00200000:3.0:1789760794.112266:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760794.112274:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760794.112277:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75309056, 75313152) 0 0 00000080:00200000:3.0:1789760794.112281:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760794.112297:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18386, 18386] 00020000:00200000:3.0:1789760794.112305:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75309056 chunk: [75309056, 75313152) 75313152, 4096 00020000:00200000:3.0:1789760794.112309:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760794.112338:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18386, 18386] 00000008:00010000:3.0:1789760794.112355:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760794.112361:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760794.112372:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75309056, 75313152) append 0 append lockpos 0 00000080:00200000:3.0:1789760794.112388:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75309056=0x47d2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760794.112457:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760794.112460:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75313152 00000020:00010000:3.0:1789760794.112468:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760794.112471:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.112474:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.112478:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760794.112481:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760794.112489:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760794.112495:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760794.112510:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760794.112927:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 75309056<>75313152 00000100:00100000:3.0:1789760794.112971:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760794.112989:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565584640:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.128915:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985694 gen 3 00000008:00010000:3.0:1789760794.128969:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:3.0:1789760794.128978:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.128989:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.129012:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:3.0:1789760794.129043:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565584640:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.129243:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18386, 18386] 00000080:00200000:1.0:1789760794.129261:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.129267:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.129282:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75313152 i_blocks 147120 00000020:00200000:1.0:1789760794.129287:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.129297:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.129342:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75313152, count: 4096 00000080:00200000:1.0:1789760794.129350:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75313152, bytes: 4096 00000080:00200000:1.0:1789760794.129361:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.129369:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.129372:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75313152, 75317248) 0 0 00000080:00200000:1.0:1789760794.129376:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.129403:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18387, 18387] 00020000:00200000:1.0:1789760794.129413:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75313152 chunk: [75313152, 75317248) 75317248, 4096 00020000:00200000:1.0:1789760794.129417:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.129437:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18387, 18387] 00000008:00010000:1.0:1789760794.129456:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.129462:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.129475:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75313152, 75317248) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.129499:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75313152=0x47d3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.129569:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.129572:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75317248 00000020:00010000:1.0:1789760794.129580:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.129584:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.129587:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a1b8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.129590:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.129593:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.129602:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.129610:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.129626:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760794.129785:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75313152<>75317248 00000100:00100000:1.0:1789760794.129836:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760794.129853:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565584768:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760794.141425:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760794.155270:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985695 gen 3 00000008:00010000:0.0:1789760794.155306:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760794.155313:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.155321:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.155376:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760794.155412:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565584768:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.155492:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18387, 18387] 00000080:00200000:1.0:1789760794.155505:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.155510:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760794.155601:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75317248 i_blocks 147128 00000020:00200000:0.0:1789760794.155607:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760794.155615:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760794.155641:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75317248, count: 4096 00000080:00200000:0.0:1789760794.155666:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75317248, bytes: 4096 00000080:00200000:0.0:1789760794.155674:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760794.155681:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760794.155683:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75317248, 75321344) 0 0 00000080:00200000:0.0:1789760794.155687:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760794.155701:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18388, 18388] 00020000:00200000:0.0:1789760794.155708:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75317248 chunk: [75317248, 75321344) 75321344, 4096 00020000:00200000:0.0:1789760794.155712:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760794.155728:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18388, 18388] 00000008:00010000:0.0:1789760794.155740:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760794.155744:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760794.155752:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75317248, 75321344) append 0 append lockpos 0 00000080:00200000:0.0:1789760794.155762:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75317248=0x47d4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760794.155804:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760794.155806:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75321344 00000020:00010000:0.0:1789760794.155813:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760794.155816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.155820:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.155823:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760794.155825:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760794.155829:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760794.155834:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760794.155846:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760794.155947:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75317248<>75321344 00000100:00100000:1.0:1789760794.155999:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565584896:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.178900:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985696 gen 3 00000008:00010000:0.0:1789760794.178937:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760794.178943:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.178952:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.178974:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760794.178996:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565584896:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.179141:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18388, 18388] 00000080:00200000:3.0:1789760794.179155:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.179160:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760794.179180:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75321344 i_blocks 147136 00000020:00200000:3.0:1789760794.179184:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760794.179195:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760794.179222:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75321344, count: 4096 00000080:00200000:3.0:1789760794.179228:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75321344, bytes: 4096 00000080:00200000:3.0:1789760794.179236:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760794.179243:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760794.179245:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75321344, 75325440) 0 0 00000080:00200000:3.0:1789760794.179249:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760794.179261:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18389, 18389] 00020000:00200000:3.0:1789760794.179268:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75321344 chunk: [75321344, 75325440) 75325440, 4096 00020000:00200000:3.0:1789760794.179272:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760794.179287:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18389, 18389] 00000008:00010000:3.0:1789760794.179304:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760794.179308:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760794.179317:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75321344, 75325440) append 0 append lockpos 0 00000080:00200000:3.0:1789760794.179328:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75321344=0x47d5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760794.179383:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760794.179386:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75325440 00000020:00010000:3.0:1789760794.179402:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760794.179405:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.179408:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.179411:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760794.179413:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760794.179420:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760794.179434:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760794.179450:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760794.179551:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64700 - object 0x240000400:5 offset 75321344<>75325440 00000100:00100000:2.0:1789760794.179718:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565585024:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.192927:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985697 gen 3 00000008:00010000:2.0:1789760794.192965:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760794.192971:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.192980:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.193053:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760794.193078:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565585024:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.193339:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18389, 18389] 00000080:00200000:3.0:1789760794.193351:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.193357:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760794.193367:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75325440 i_blocks 147144 00000020:00200000:3.0:1789760794.193371:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760794.193378:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760794.193416:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75325440, count: 4096 00000080:00200000:3.0:1789760794.193423:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75325440, bytes: 4096 00000080:00200000:3.0:1789760794.193503:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760794.193511:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760794.193513:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75325440, 75329536) 0 0 00000080:00200000:3.0:1789760794.193517:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760794.193530:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18390, 18390] 00020000:00200000:3.0:1789760794.193537:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75325440 chunk: [75325440, 75329536) 75329536, 4096 00020000:00200000:3.0:1789760794.193541:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760794.193555:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18390, 18390] 00000008:00010000:3.0:1789760794.193567:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760794.193572:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760794.193581:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75325440, 75329536) append 0 append lockpos 0 00000080:00200000:3.0:1789760794.193590:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75325440=0x47d6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760794.193631:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760794.193634:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75329536 00000020:00010000:3.0:1789760794.193640:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760794.193777:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.193781:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.193784:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760794.193787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760794.193794:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760794.193800:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760794.193816:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760794.193905:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67480 - object 0x240000400:5 offset 75325440<>75329536 00000100:00100000:2.0:1789760794.193965:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565585152:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.208230:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985698 gen 3 00000008:00010000:2.0:1789760794.208267:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760794.208273:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.208282:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.208360:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760794.208386:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565585152:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.209155:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18390, 18390] 00000080:00200000:3.0:1789760794.209168:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.209172:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760794.209185:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75329536 i_blocks 147152 00000020:00200000:3.0:1789760794.209189:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760794.209196:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760794.209274:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75329536, count: 4096 00000080:00200000:3.0:1789760794.209281:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75329536, bytes: 4096 00000080:00200000:3.0:1789760794.209291:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760794.209298:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760794.209300:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75329536, 75333632) 0 0 00000080:00200000:3.0:1789760794.209304:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760794.209317:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18391, 18391] 00020000:00200000:3.0:1789760794.209324:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75329536 chunk: [75329536, 75333632) 75333632, 4096 00020000:00200000:3.0:1789760794.209327:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760794.209342:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18391, 18391] 00000008:00010000:3.0:1789760794.209356:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760794.209361:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760794.209369:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75329536, 75333632) append 0 append lockpos 0 00000080:00200000:3.0:1789760794.209379:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75329536=0x47d7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760794.209468:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760794.209471:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75333632 00000020:00010000:3.0:1789760794.209526:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:3.0:1789760794.209530:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.209534:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760794.209536:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760794.209539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760794.209546:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760794.209552:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760794.209570:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760794.211175:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67800 - object 0x240000400:5 offset 75329536<>75333632 00000100:00100000:2.0:1789760794.211272:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760794.211354:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565585280:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.222614:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985699 gen 3 00000008:00010000:2.0:1789760794.222715:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760794.222724:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.222738:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.222759:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760794.222795:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565585280:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760794.222839:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18391, 18391] 00000080:00200000:0.0:1789760794.222864:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760794.222870:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760794.222886:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75333632 i_blocks 147160 00000020:00200000:0.0:1789760794.222891:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760794.222905:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760794.222942:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75333632, count: 4096 00000080:00200000:0.0:1789760794.222950:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75333632, bytes: 4096 00000080:00200000:0.0:1789760794.222964:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760794.222972:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760794.222974:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75333632, 75337728) 0 0 00000080:00200000:0.0:1789760794.222978:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760794.222994:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18392, 18392] 00020000:00200000:0.0:1789760794.223003:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75333632 chunk: [75333632, 75337728) 75337728, 4096 00020000:00200000:0.0:1789760794.223009:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760794.223027:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18392, 18392] 00000008:00010000:0.0:1789760794.223042:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760794.223048:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760794.223058:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75333632, 75337728) append 0 append lockpos 0 00000080:00200000:0.0:1789760794.223070:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75333632=0x47d8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760794.223128:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760794.223131:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75337728 00000020:00010000:0.0:1789760794.223139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760794.223142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.223145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.223148:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760794.223151:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760794.223160:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760794.223166:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760794.223185:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760794.223307:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75333632<>75337728 00000100:00100000:1.0:1789760794.223589:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565585408:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.238630:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985700 gen 3 00000008:00010000:0.0:1789760794.238701:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760794.238708:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.238718:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.238744:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760794.238766:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565585408:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.238950:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18392, 18392] 00000080:00200000:2.0:1789760794.238963:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.238968:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.238980:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75337728 i_blocks 147168 00000020:00200000:2.0:1789760794.238983:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.238992:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.239018:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75337728, count: 4096 00000080:00200000:2.0:1789760794.239024:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75337728, bytes: 4096 00000080:00200000:2.0:1789760794.239032:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.239039:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.239042:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75337728, 75341824) 0 0 00000080:00200000:2.0:1789760794.239045:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.239059:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18393, 18393] 00020000:00200000:2.0:1789760794.239066:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75337728 chunk: [75337728, 75341824) 75341824, 4096 00020000:00200000:2.0:1789760794.239070:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.239084:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18393, 18393] 00000008:00010000:2.0:1789760794.239098:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.239103:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.239111:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75337728, 75341824) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.239122:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75337728=0x47d9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.239166:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.239169:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75341824 00000020:00010000:2.0:1789760794.239176:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.239179:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.239182:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.239185:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.239188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.239194:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.239199:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.239213:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760794.239308:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 75337728<>75341824 00000100:00100000:3.0:1789760794.239541:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565585536:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.250638:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985701 gen 3 00000008:00010000:3.0:1789760794.250728:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:3.0:1789760794.250737:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.250749:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.250769:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:3.0:1789760794.250802:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565585536:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.250827:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18393, 18393] 00000080:00200000:2.0:1789760794.250858:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.250864:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.250884:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75341824 i_blocks 147176 00000020:00200000:2.0:1789760794.250888:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.250901:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.250937:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75341824, count: 4096 00000080:00200000:2.0:1789760794.250945:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75341824, bytes: 4096 00000080:00200000:2.0:1789760794.250955:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.250962:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.250965:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75341824, 75345920) 0 0 00000080:00200000:2.0:1789760794.250968:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.250982:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18394, 18394] 00020000:00200000:2.0:1789760794.250989:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75341824 chunk: [75341824, 75345920) 75345920, 4096 00020000:00200000:2.0:1789760794.250993:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.251009:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18394, 18394] 00000008:00010000:2.0:1789760794.251022:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.251027:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.251046:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75341824, 75345920) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.251057:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75341824=0x47da000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.251110:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.251113:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75345920 00000020:00010000:2.0:1789760794.251120:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.251123:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.251125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.251128:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.251131:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.251138:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.251143:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.251162:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760794.251258:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 75341824<>75345920 00000100:00100000:3.0:1789760794.251351:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565585664:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.268047:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985702 gen 3 00000008:00010000:3.0:1789760794.268088:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760794.268096:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.268105:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.268133:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760794.268157:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565585664:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.268387:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18394, 18394] 00000080:00200000:2.0:1789760794.268407:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.268412:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.268423:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75345920 i_blocks 147184 00000020:00200000:2.0:1789760794.268426:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.268435:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.268459:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75345920, count: 4096 00000080:00200000:2.0:1789760794.268465:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75345920, bytes: 4096 00000080:00200000:2.0:1789760794.268472:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.268479:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.268481:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75345920, 75350016) 0 0 00000080:00200000:2.0:1789760794.268485:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.268497:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18395, 18395] 00020000:00200000:2.0:1789760794.268504:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75345920 chunk: [75345920, 75350016) 75350016, 4096 00020000:00200000:2.0:1789760794.268508:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.268522:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18395, 18395] 00000008:00010000:2.0:1789760794.268535:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.268540:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.268548:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75345920, 75350016) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.268558:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75345920=0x47db000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.268597:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.268600:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75350016 00000020:00010000:2.0:1789760794.268607:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.268610:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.268612:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.268615:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.268618:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.268624:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.268628:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.268915:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760794.269009:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 75345920<>75350016 00000100:00100000:2.0:1789760794.269053:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760794.269068:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565585792:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.285616:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985703 gen 3 00000008:00010000:2.0:1789760794.285678:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760794.285684:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.285693:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.285750:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760794.285780:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565585792:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.286102:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18395, 18395] 00000080:00200000:3.0:1789760794.286116:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.286122:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.286214:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75350016 i_blocks 147192 00000020:00200000:2.0:1789760794.286219:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.286226:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.286250:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75350016, count: 4096 00000080:00200000:2.0:1789760794.286256:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75350016, bytes: 4096 00000080:00200000:2.0:1789760794.286263:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.286270:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.286273:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75350016, 75354112) 0 0 00000080:00200000:2.0:1789760794.286277:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.286288:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18396, 18396] 00020000:00200000:2.0:1789760794.286295:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75350016 chunk: [75350016, 75354112) 75354112, 4096 00020000:00200000:2.0:1789760794.286299:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.286373:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18396, 18396] 00000008:00010000:2.0:1789760794.286386:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.286576:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.286589:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75350016, 75354112) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.286600:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75350016=0x47dc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.286822:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.286826:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75354112 00000020:00010000:2.0:1789760794.286834:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.286837:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.286842:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.286845:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.286847:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.286853:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.286859:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.286900:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760794.287125:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 75350016<>75354112 00000100:00100000:3.0:1789760794.287185:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565585920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.300973:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985704 gen 3 00000008:00010000:3.0:1789760794.301024:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760794.301032:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.301043:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.301064:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760794.301096:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565585920:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760794.301452:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18396, 18396] 00000080:00200000:0.0:1789760794.301474:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760794.301479:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760794.301492:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75354112 i_blocks 147200 00000020:00200000:0.0:1789760794.301496:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760794.301509:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760794.301535:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75354112, count: 4096 00000080:00200000:0.0:1789760794.301542:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75354112, bytes: 4096 00000080:00200000:0.0:1789760794.301552:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760794.301560:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760794.301562:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75354112, 75358208) 0 0 00000080:00200000:0.0:1789760794.301565:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760794.301578:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18397, 18397] 00020000:00200000:0.0:1789760794.301585:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75354112 chunk: [75354112, 75358208) 75358208, 4096 00020000:00200000:0.0:1789760794.301588:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760794.301602:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18397, 18397] 00000008:00010000:0.0:1789760794.301614:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760794.301618:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760794.301626:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75354112, 75358208) append 0 append lockpos 0 00000080:00200000:0.0:1789760794.301636:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75354112=0x47dd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760794.301726:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760794.301729:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75358208 00000020:00010000:0.0:1789760794.301735:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760794.301739:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.301741:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.301744:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760794.301746:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760794.301762:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760794.301768:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760794.301787:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760794.301884:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75354112<>75358208 00000100:00100000:0.0:1789760794.301922:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760794.301939:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565586048:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.314699:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985705 gen 3 00000008:00010000:0.0:1789760794.314733:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760794.314813:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.314821:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.314901:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760794.314938:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565586048:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.315151:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18397, 18397] 00000080:00200000:2.0:1789760794.315163:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.315168:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.315179:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75358208 i_blocks 147208 00000020:00200000:2.0:1789760794.315182:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.315192:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.315216:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75358208, count: 4096 00000080:00200000:2.0:1789760794.315222:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75358208, bytes: 4096 00000080:00200000:2.0:1789760794.315229:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.315237:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.315239:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75358208, 75362304) 0 0 00000080:00200000:2.0:1789760794.315242:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.315256:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18398, 18398] 00020000:00200000:2.0:1789760794.315263:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75358208 chunk: [75358208, 75362304) 75362304, 4096 00020000:00200000:2.0:1789760794.315266:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.315281:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18398, 18398] 00000008:00010000:2.0:1789760794.315294:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.315299:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.315307:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75358208, 75362304) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.315317:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75358208=0x47de000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.315436:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.315439:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75362304 00000020:00010000:2.0:1789760794.315446:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.315450:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.315452:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.315455:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.315458:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.315464:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.315469:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.315483:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760794.315678:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 75358208<>75362304 00000100:00100000:3.0:1789760794.315765:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565586176:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.327759:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985706 gen 3 00000008:00010000:2.0:1789760794.327795:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760794.327801:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.327809:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.327831:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760794.327856:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565586176:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760794.327871:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18398, 18398] 00000080:00200000:0.0:1789760794.327886:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760794.327891:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760794.327903:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75362304 i_blocks 147216 00000020:00200000:0.0:1789760794.327906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760794.327914:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760794.327938:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75362304, count: 4096 00000080:00200000:0.0:1789760794.327945:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75362304, bytes: 4096 00000080:00200000:0.0:1789760794.327952:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760794.327960:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760794.327961:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75362304, 75366400) 0 0 00000080:00200000:0.0:1789760794.327965:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760794.327978:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18399, 18399] 00020000:00200000:0.0:1789760794.327984:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75362304 chunk: [75362304, 75366400) 75366400, 4096 00020000:00200000:0.0:1789760794.327988:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760794.328002:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18399, 18399] 00000008:00010000:0.0:1789760794.328015:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760794.328020:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760794.328028:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75362304, 75366400) append 0 append lockpos 0 00000080:00200000:0.0:1789760794.328039:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75362304=0x47df000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760794.328080:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760794.328083:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75366400 00000020:00010000:0.0:1789760794.328089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760794.328093:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.328096:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.328099:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760794.328102:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760794.328108:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760794.328113:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760794.328126:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760794.328213:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75362304<>75366400 00000100:00100000:1.0:1789760794.328339:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565586304:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.343601:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985707 gen 3 00000008:00010000:0.0:1789760794.343638:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760794.343669:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.343678:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.343702:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760794.343724:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565586304:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.343823:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18399, 18399] 00000080:00200000:2.0:1789760794.343840:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.343844:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.343856:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75366400 i_blocks 147224 00000020:00200000:2.0:1789760794.343859:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.343868:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.343964:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75366400, count: 4096 00000080:00200000:2.0:1789760794.343972:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75366400, bytes: 4096 00000080:00200000:2.0:1789760794.343980:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.343988:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.343990:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75366400, 75370496) 0 0 00000080:00200000:2.0:1789760794.343994:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.344006:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18400, 18400] 00020000:00200000:2.0:1789760794.344013:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75366400 chunk: [75366400, 75370496) 75370496, 4096 00020000:00200000:2.0:1789760794.344017:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.344030:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18400, 18400] 00000008:00010000:2.0:1789760794.344108:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.344114:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.344124:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75366400, 75370496) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.344135:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75366400=0x47e0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.344177:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.344180:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75370496 00000020:00010000:2.0:1789760794.344186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.344189:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.344192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.344196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.344198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.344204:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.344209:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.344223:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760794.344310:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 75366400<>75370496 00000100:00100000:3.0:1789760794.344500:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565586432:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.357907:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985708 gen 3 00000008:00010000:2.0:1789760794.357942:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760794.357948:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.357956:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.358032:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760794.358066:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565586432:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760794.358274:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18400, 18400] 00000080:00200000:0.0:1789760794.358288:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760794.358293:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760794.358305:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75370496 i_blocks 147232 00000020:00200000:0.0:1789760794.358308:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760794.358317:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760794.358410:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75370496, count: 4096 00000080:00200000:0.0:1789760794.358418:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75370496, bytes: 4096 00000080:00200000:0.0:1789760794.358426:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760794.358433:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760794.358436:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75370496, 75374592) 0 0 00000080:00200000:0.0:1789760794.358440:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760794.358453:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18401, 18401] 00020000:00200000:0.0:1789760794.358459:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75370496 chunk: [75370496, 75374592) 75374592, 4096 00020000:00200000:0.0:1789760794.358463:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760794.358476:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18401, 18401] 00000008:00010000:0.0:1789760794.358490:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760794.358494:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760794.358502:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75370496, 75374592) append 0 append lockpos 0 00000080:00200000:0.0:1789760794.358512:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75370496=0x47e1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760794.358557:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760794.358560:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75374592 00000020:00010000:0.0:1789760794.358566:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760794.358570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.358573:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.358576:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760794.358579:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760794.358585:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760794.358591:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760794.358605:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760794.358730:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75370496<>75374592 00000100:00100000:0.0:1789760794.358761:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760794.358774:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565586560:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.378478:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985709 gen 3 00000008:00010000:0.0:1789760794.378512:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760794.378518:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.378526:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.378592:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760794.378615:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565586560:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.379086:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18401, 18401] 00000080:00200000:2.0:1789760794.379100:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.379104:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.379116:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75374592 i_blocks 147240 00000020:00200000:2.0:1789760794.379119:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.379128:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.379153:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75374592, count: 4096 00000080:00200000:2.0:1789760794.379160:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75374592, bytes: 4096 00000080:00200000:2.0:1789760794.379168:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.379175:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.379178:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75374592, 75378688) 0 0 00000080:00200000:2.0:1789760794.379182:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.379261:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18402, 18402] 00020000:00200000:2.0:1789760794.379268:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75374592 chunk: [75374592, 75378688) 75378688, 4096 00020000:00200000:2.0:1789760794.379272:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.379286:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18402, 18402] 00000008:00010000:2.0:1789760794.379298:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.379304:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.379313:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75374592, 75378688) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.379323:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75374592=0x47e2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.379367:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.379370:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75378688 00000020:00010000:2.0:1789760794.379376:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.379379:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.379382:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.379385:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.379387:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.379403:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.379409:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.379425:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760794.379515:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 75374592<>75378688 00000100:00100000:2.0:1789760794.379558:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760794.379572:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565586688:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.412639:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985710 gen 3 00000008:00010000:2.0:1789760794.412812:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760794.412819:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.412828:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.412897:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760794.412921:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565586688:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760794.413686:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18402, 18402] 00000080:00200000:0.0:1789760794.413700:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760794.413706:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760794.413717:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75378688 i_blocks 147248 00000020:00200000:0.0:1789760794.413721:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760794.413730:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760794.413825:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75378688, count: 4096 00000080:00200000:0.0:1789760794.413832:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75378688, bytes: 4096 00000080:00200000:0.0:1789760794.413842:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760794.413849:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760794.413851:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75378688, 75382784) 0 0 00000080:00200000:0.0:1789760794.413854:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760794.413866:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18403, 18403] 00020000:00200000:0.0:1789760794.413874:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75378688 chunk: [75378688, 75382784) 75382784, 4096 00020000:00200000:0.0:1789760794.413877:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760794.413894:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18403, 18403] 00000008:00010000:0.0:1789760794.413908:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760794.413912:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760794.413921:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75378688, 75382784) append 0 append lockpos 0 00000080:00200000:0.0:1789760794.413931:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75378688=0x47e3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760794.413977:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760794.413980:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75382784 00000020:00010000:0.0:1789760794.413986:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760794.413989:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.413992:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.413995:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760794.413997:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760794.414003:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760794.414008:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760794.414025:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760794.414117:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75378688<>75382784 00000100:00100000:1.0:1789760794.414202:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565586816:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.428531:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985711 gen 3 00000008:00010000:0.0:1789760794.428572:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760794.428578:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.428586:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.428814:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18403, 18403] 00000080:00200000:2.0:1789760794.428828:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.428833:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.428844:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75382784 i_blocks 147256 00000020:00200000:2.0:1789760794.428848:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.428857:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.428881:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75382784, count: 4096 00000080:00200000:2.0:1789760794.428888:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75382784, bytes: 4096 00000080:00200000:2.0:1789760794.428897:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.428904:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.428906:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75382784, 75386880) 0 0 00000080:00200000:2.0:1789760794.428910:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.428923:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18404, 18404] 00020000:00200000:2.0:1789760794.428930:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75382784 chunk: [75382784, 75386880) 75386880, 4096 00020000:00200000:2.0:1789760794.428934:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.428947:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18404, 18404] 00000008:00010000:2.0:1789760794.428961:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.428966:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.428973:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75382784, 75386880) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.428982:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75382784=0x47e4000, pages 1 (max 131072), lockless, parallel 00000008:00010000:0.0:1789760794.429027:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760794.429049:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565586816:192.168.203.147@tcp:4:dd.0 00000020:00200000:2.0:1789760794.429232:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.429235:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75386880 00000020:00010000:2.0:1789760794.429243:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.429246:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.429249:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.429252:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.429255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.429262:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.429268:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.429281:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760794.429375:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 75382784<>75386880 00000100:00100000:3.0:1789760794.429460:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565586944:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.443805:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985712 gen 3 00000008:00010000:3.0:1789760794.443841:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760794.443847:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.443855:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.443871:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760794.443896:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565586944:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.443906:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18404, 18404] 00000080:00200000:2.0:1789760794.443921:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.443927:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.443938:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75386880 i_blocks 147264 00000020:00200000:2.0:1789760794.443942:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.443951:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.443976:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75386880, count: 4096 00000080:00200000:2.0:1789760794.443982:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75386880, bytes: 4096 00000080:00200000:2.0:1789760794.443988:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.443995:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.443997:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75386880, 75390976) 0 0 00000080:00200000:2.0:1789760794.444001:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.444014:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18405, 18405] 00020000:00200000:2.0:1789760794.444021:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75386880 chunk: [75386880, 75390976) 75390976, 4096 00020000:00200000:2.0:1789760794.444024:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.444038:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18405, 18405] 00000008:00010000:2.0:1789760794.444051:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.444055:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.444063:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75386880, 75390976) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.444073:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75386880=0x47e5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.444114:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.444117:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75390976 00000020:00010000:2.0:1789760794.444123:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.444126:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.444129:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.444132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.444135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.444140:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.444197:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.444212:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760794.444307:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 75386880<>75390976 00000100:00100000:3.0:1789760794.444426:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565587072:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.455934:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985713 gen 3 00000008:00010000:3.0:1789760794.455971:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760794.455977:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.455985:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.456011:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760794.456035:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565587072:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.456049:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18405, 18405] 00000080:00200000:2.0:1789760794.456063:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.456074:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.456087:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75390976 i_blocks 147272 00000020:00200000:2.0:1789760794.456091:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.456100:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.456126:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75390976, count: 4096 00000080:00200000:2.0:1789760794.456132:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75390976, bytes: 4096 00000080:00200000:2.0:1789760794.456139:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.456146:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.456148:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75390976, 75395072) 0 0 00000080:00200000:2.0:1789760794.456151:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.456164:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18406, 18406] 00020000:00200000:2.0:1789760794.456171:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75390976 chunk: [75390976, 75395072) 75395072, 4096 00020000:00200000:2.0:1789760794.456175:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.456302:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18406, 18406] 00000008:00010000:2.0:1789760794.456317:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.456322:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.456330:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75390976, 75395072) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.456340:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75390976=0x47e6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.456381:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.456385:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75395072 00000020:00010000:2.0:1789760794.456401:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.456405:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.456408:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.456410:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.456413:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.456419:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.456424:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.456439:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760794.456530:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 75390976<>75395072 00000100:00100000:3.0:1789760794.456681:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565587200:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.477135:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985714 gen 3 00000008:00010000:3.0:1789760794.477318:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760794.477596:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.477606:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.477626:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760794.477749:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565587200:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.478430:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18406, 18406] 00000080:00200000:2.0:1789760794.478442:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.478448:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.478461:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75395072 i_blocks 147280 00000020:00200000:2.0:1789760794.478464:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.478472:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.478496:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75395072, count: 4096 00000080:00200000:2.0:1789760794.478503:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75395072, bytes: 4096 00000080:00200000:2.0:1789760794.478512:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.478519:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.478521:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75395072, 75399168) 0 0 00000080:00200000:2.0:1789760794.478524:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.478536:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18407, 18407] 00020000:00200000:2.0:1789760794.478542:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75395072 chunk: [75395072, 75399168) 75399168, 4096 00020000:00200000:2.0:1789760794.478546:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.478560:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18407, 18407] 00000008:00010000:2.0:1789760794.478572:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.478577:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.478586:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75395072, 75399168) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.478596:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75395072=0x47e7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.478640:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.478886:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75399168 00000020:00010000:2.0:1789760794.478895:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.478898:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.478901:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.478904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.478907:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.478914:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.478919:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.478934:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760794.479590:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 75395072<>75399168 00000100:00100000:3.0:1789760794.480022:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565587328:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.493103:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985715 gen 3 00000008:00010000:3.0:1789760794.493138:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760794.493143:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.493150:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.493170:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760794.493195:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565587328:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.493220:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18407, 18407] 00000080:00200000:2.0:1789760794.493231:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.493235:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.493245:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75399168 i_blocks 147288 00000020:00200000:2.0:1789760794.493247:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.493255:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.493275:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75399168, count: 4096 00000080:00200000:2.0:1789760794.493280:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75399168, bytes: 4096 00000080:00200000:2.0:1789760794.493285:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.493291:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.493293:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75399168, 75403264) 0 0 00000080:00200000:2.0:1789760794.493296:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.493306:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18408, 18408] 00020000:00200000:2.0:1789760794.493312:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75399168 chunk: [75399168, 75403264) 75403264, 4096 00020000:00200000:2.0:1789760794.493315:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.493327:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18408, 18408] 00000008:00010000:2.0:1789760794.493337:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.493340:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.493347:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75399168, 75403264) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.493356:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75399168=0x47e8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.493404:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.493407:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75403264 00000020:00010000:2.0:1789760794.493412:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.493414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.493416:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.493418:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.493420:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.493424:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.493428:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.493441:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760794.493522:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 75399168<>75403264 00000100:00100000:3.0:1789760794.493577:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760794.493591:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565587456:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.503457:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985716 gen 3 00000008:00010000:3.0:1789760794.503493:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760794.503499:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.503508:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.503529:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760794.503553:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565587456:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.503668:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18408, 18408] 00000080:00200000:2.0:1789760794.503785:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.503791:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.503803:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75403264 i_blocks 147296 00000020:00200000:2.0:1789760794.503806:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.503813:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.503838:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75403264, count: 4096 00000080:00200000:2.0:1789760794.503844:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75403264, bytes: 4096 00000080:00200000:2.0:1789760794.503851:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.503858:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.503860:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75403264, 75407360) 0 0 00000080:00200000:2.0:1789760794.503864:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.503877:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18409, 18409] 00020000:00200000:2.0:1789760794.503883:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75403264 chunk: [75403264, 75407360) 75407360, 4096 00020000:00200000:2.0:1789760794.503887:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.503903:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18409, 18409] 00000008:00010000:2.0:1789760794.503915:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.503920:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.503928:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75403264, 75407360) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.503938:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75403264=0x47e9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.503977:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.503980:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75407360 00000020:00010000:2.0:1789760794.503986:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.503990:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.503993:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.503996:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.503999:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.504005:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.504011:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.504030:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760794.504122:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 75403264<>75407360 00000100:00100000:3.0:1789760794.504190:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565587584:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.521973:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985717 gen 3 00000008:00010000:3.0:1789760794.522010:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760794.522016:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.522025:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.522046:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760794.522072:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565587584:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.522268:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18409, 18409] 00000080:00200000:2.0:1789760794.522282:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.522288:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.522309:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75407360 i_blocks 147304 00000020:00200000:2.0:1789760794.522312:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.522321:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.522349:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75407360, count: 4096 00000080:00200000:2.0:1789760794.522355:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75407360, bytes: 4096 00000080:00200000:2.0:1789760794.522366:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.522374:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.522376:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75407360, 75411456) 0 0 00000080:00200000:2.0:1789760794.522380:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.522403:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18410, 18410] 00020000:00200000:2.0:1789760794.522410:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75407360 chunk: [75407360, 75411456) 75411456, 4096 00020000:00200000:2.0:1789760794.522413:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.522494:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18410, 18410] 00000008:00010000:2.0:1789760794.522509:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.522513:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.522521:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75407360, 75411456) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.522531:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75407360=0x47ea000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.522626:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.522629:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75411456 00000020:00010000:2.0:1789760794.522636:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.522639:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.522642:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.522664:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.522666:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.522673:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.522679:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.522694:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760794.522782:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 75407360<>75411456 00000100:00100000:2.0:1789760794.522825:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565587712:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.545271:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985718 gen 3 00000008:00010000:2.0:1789760794.545310:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:2.0:1789760794.545317:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.545385:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.545457:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:2.0:1789760794.545478:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565587712:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760794.545552:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18410, 18410] 00000080:00200000:3.0:1789760794.545564:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760794.545569:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.545675:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75411456 i_blocks 147312 00000020:00200000:2.0:1789760794.545680:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.545687:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.545711:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75411456, count: 4096 00000080:00200000:2.0:1789760794.545717:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75411456, bytes: 4096 00000080:00200000:2.0:1789760794.545726:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.545733:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.545735:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75411456, 75415552) 0 0 00000080:00200000:2.0:1789760794.545740:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.545752:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18411, 18411] 00020000:00200000:2.0:1789760794.545758:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75411456 chunk: [75411456, 75415552) 75415552, 4096 00020000:00200000:2.0:1789760794.545762:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.545775:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18411, 18411] 00000008:00010000:2.0:1789760794.545788:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.545793:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.545802:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75411456, 75415552) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.545812:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75411456=0x47eb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.545849:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.545851:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75415552 00000020:00010000:2.0:1789760794.545858:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.545861:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.545864:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.545868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.545870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.545876:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.545881:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.546090:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760794.546180:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 75411456<>75415552 00000100:00100000:3.0:1789760794.546237:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565587840:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.561957:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985719 gen 3 00000008:00010000:3.0:1789760794.561994:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:3.0:1789760794.562001:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.562010:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.562031:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:3.0:1789760794.562056:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565587840:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.562144:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18411, 18411] 00000080:00200000:1.0:1789760794.562158:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.562163:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.562174:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75415552 i_blocks 147320 00000020:00200000:1.0:1789760794.562179:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.562187:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.562215:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75415552, count: 4096 00000080:00200000:1.0:1789760794.562222:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75415552, bytes: 4096 00000080:00200000:1.0:1789760794.562230:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.562237:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.562239:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75415552, 75419648) 0 0 00000080:00200000:1.0:1789760794.562243:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.562258:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18412, 18412] 00020000:00200000:1.0:1789760794.562264:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75415552 chunk: [75415552, 75419648) 75419648, 4096 00020000:00200000:1.0:1789760794.562268:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.562284:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18412, 18412] 00000008:00010000:1.0:1789760794.562299:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.562304:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.562313:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75415552, 75419648) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.562323:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75415552=0x47ec000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.562383:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.562386:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75419648 00000020:00010000:1.0:1789760794.562401:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.562405:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.562407:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.562411:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.562413:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.562420:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.562426:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.562441:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760794.562541:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 75415552<>75419648 00000100:00100000:0.0:1789760794.562721:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565587968:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.579941:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985720 gen 3 00000008:00010000:0.0:1789760794.579977:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760794.579984:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.579992:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.580010:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760794.580033:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565587968:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.580125:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18412, 18412] 00000080:00200000:1.0:1789760794.580139:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.580143:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.580154:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75419648 i_blocks 147328 00000020:00200000:1.0:1789760794.580159:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.580166:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.580191:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75419648, count: 4096 00000080:00200000:1.0:1789760794.580198:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75419648, bytes: 4096 00000080:00200000:1.0:1789760794.580206:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.580214:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.580216:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75419648, 75423744) 0 0 00000080:00200000:1.0:1789760794.580220:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.580232:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18413, 18413] 00020000:00200000:1.0:1789760794.580239:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75419648 chunk: [75419648, 75423744) 75423744, 4096 00020000:00200000:1.0:1789760794.580243:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.580257:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18413, 18413] 00000008:00010000:1.0:1789760794.580271:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.580275:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.580284:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75419648, 75423744) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.580295:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75419648=0x47ed000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.580348:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.580351:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75423744 00000020:00010000:1.0:1789760794.580359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.580362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.580364:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.580367:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.580370:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.580622:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.580629:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.580676:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760794.580968:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 75419648<>75423744 00000100:00100000:1.0:1789760794.581010:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760794.581025:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565588096:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.593904:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985721 gen 3 00000008:00010000:0.0:1789760794.593939:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760794.593946:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.593963:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.594035:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760794.594057:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565588096:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.596588:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18413, 18413] 00000080:00200000:2.0:1789760794.596608:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.596613:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.596627:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75423744 i_blocks 147336 00000020:00200000:2.0:1789760794.596631:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.596869:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.596897:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75423744, count: 4096 00000080:00200000:2.0:1789760794.596904:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75423744, bytes: 4096 00000080:00200000:2.0:1789760794.596915:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.596922:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.596924:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75423744, 75427840) 0 0 00000080:00200000:2.0:1789760794.596928:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.596940:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18414, 18414] 00020000:00200000:2.0:1789760794.596947:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75423744 chunk: [75423744, 75427840) 75427840, 4096 00020000:00200000:2.0:1789760794.596951:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.596970:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18414, 18414] 00000008:00010000:2.0:1789760794.596983:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.596988:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.596997:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75423744, 75427840) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.597009:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75423744=0x47ee000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.597061:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.597063:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75427840 00000020:00010000:2.0:1789760794.597070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.597074:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.597077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.597080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.597082:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.597089:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.597095:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.597117:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760794.597214:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 75423744<>75427840 00000100:00100000:3.0:1789760794.597295:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565588224:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.608606:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985722 gen 3 00000008:00010000:3.0:1789760794.608676:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:3.0:1789760794.608684:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.608691:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.608710:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:3.0:1789760794.608734:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565588224:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.608925:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18414, 18414] 00000080:00200000:2.0:1789760794.608939:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.608945:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.608957:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75427840 i_blocks 147344 00000020:00200000:2.0:1789760794.608960:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.608969:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.608995:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75427840, count: 4096 00000080:00200000:2.0:1789760794.609001:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75427840, bytes: 4096 00000080:00200000:2.0:1789760794.609009:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.609016:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.609018:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75427840, 75431936) 0 0 00000080:00200000:2.0:1789760794.609022:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.609035:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18415, 18415] 00020000:00200000:2.0:1789760794.609042:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75427840 chunk: [75427840, 75431936) 75431936, 4096 00020000:00200000:2.0:1789760794.609046:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.609061:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18415, 18415] 00000008:00010000:2.0:1789760794.609074:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.609078:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.609087:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75427840, 75431936) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.609097:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75427840=0x47ef000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.609137:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.609140:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75431936 00000020:00010000:2.0:1789760794.609147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.609150:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.609153:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.609156:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.609159:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.609165:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.609171:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.609185:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760794.609278:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 75427840<>75431936 00000100:00100000:3.0:1789760794.609443:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565588352:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.617444:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985723 gen 3 00000008:00010000:3.0:1789760794.617495:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760794.617502:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760794.617511:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760794.617529:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760794.617553:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565588352:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.617766:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18415, 18415] 00000080:00200000:2.0:1789760794.617780:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.617785:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.617797:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75431936 i_blocks 147352 00000020:00200000:2.0:1789760794.617800:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.617809:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.617836:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75431936, count: 4096 00000080:00200000:2.0:1789760794.617843:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75431936, bytes: 4096 00000080:00200000:2.0:1789760794.617850:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.617858:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.617860:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75431936, 75436032) 0 0 00000080:00200000:2.0:1789760794.617863:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.617876:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18416, 18416] 00020000:00200000:2.0:1789760794.617883:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75431936 chunk: [75431936, 75436032) 75436032, 4096 00020000:00200000:2.0:1789760794.617887:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.617902:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18416, 18416] 00000008:00010000:2.0:1789760794.617916:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.617921:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.617930:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75431936, 75436032) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.617940:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75431936=0x47f0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.617982:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.617984:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75436032 00000020:00010000:2.0:1789760794.617991:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.617994:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.617997:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.618001:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.618003:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.618009:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.618015:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.618030:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760794.618127:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 75431936<>75436032 00000100:00100000:3.0:1789760794.618190:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565588480:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760794.636753:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985724 gen 3 00000008:00010000:2.0:1789760794.636932:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760794.637104:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.637115:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.637134:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760794.637157:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565588480:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760794.637252:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18416, 18416] 00000080:00200000:0.0:1789760794.637263:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760794.637268:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760794.637280:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75436032 i_blocks 147360 00000020:00200000:0.0:1789760794.637283:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760794.637291:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760794.637315:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75436032, count: 4096 00000080:00200000:0.0:1789760794.637321:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75436032, bytes: 4096 00000080:00200000:0.0:1789760794.637331:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760794.637337:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760794.637339:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75436032, 75440128) 0 0 00000080:00200000:0.0:1789760794.637343:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760794.637354:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18417, 18417] 00020000:00200000:0.0:1789760794.637361:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75436032 chunk: [75436032, 75440128) 75440128, 4096 00020000:00200000:0.0:1789760794.637364:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760794.637633:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18417, 18417] 00000008:00010000:0.0:1789760794.637669:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760794.637673:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760794.637683:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75436032, 75440128) append 0 append lockpos 0 00000080:00200000:0.0:1789760794.637693:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75436032=0x47f1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760794.637734:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760794.637737:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75440128 00000020:00010000:0.0:1789760794.637743:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760794.637747:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.637749:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.637752:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760794.637755:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760794.637760:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760794.637765:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760794.637779:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760794.637933:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75436032<>75440128 00000100:00100000:1.0:1789760794.638186:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565588608:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.652304:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985725 gen 3 00000008:00010000:0.0:1789760794.652343:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760794.652349:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.652357:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.652575:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760794.652594:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565588608:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.652731:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18417, 18417] 00000080:00200000:2.0:1789760794.652743:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.652748:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.652759:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75440128 i_blocks 147368 00000020:00200000:2.0:1789760794.652762:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.652771:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.652795:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75440128, count: 4096 00000080:00200000:2.0:1789760794.652802:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75440128, bytes: 4096 00000080:00200000:2.0:1789760794.652811:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.652828:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.652831:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75440128, 75444224) 0 0 00000080:00200000:2.0:1789760794.652834:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.652846:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18418, 18418] 00020000:00200000:2.0:1789760794.652853:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75440128 chunk: [75440128, 75444224) 75444224, 4096 00020000:00200000:2.0:1789760794.652857:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.652873:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18418, 18418] 00000008:00010000:2.0:1789760794.652886:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.652891:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.652901:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75440128, 75444224) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.652911:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75440128=0x47f2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.652949:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.652952:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75444224 00000020:00010000:2.0:1789760794.652958:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.652961:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.652964:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.652967:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.652969:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.652976:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.652981:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760794.653431:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:0.0:1789760794.653550:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75440128<>75444224 00000100:00100000:0.0:1789760794.653587:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760794.653602:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565588736:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.669264:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985726 gen 3 00000008:00010000:0.0:1789760794.669299:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:0.0:1789760794.669306:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.669314:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.669345:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:0.0:1789760794.669369:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565588736:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.669716:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18418, 18418] 00000080:00200000:2.0:1789760794.669731:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.669747:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.669760:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75444224 i_blocks 147376 00000020:00200000:2.0:1789760794.669764:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.669773:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.669802:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75444224, count: 4096 00000080:00200000:2.0:1789760794.669810:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75444224, bytes: 4096 00000080:00200000:2.0:1789760794.669821:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.669829:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.669831:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75444224, 75448320) 0 0 00000080:00200000:2.0:1789760794.669836:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.669851:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18419, 18419] 00020000:00200000:2.0:1789760794.669859:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75444224 chunk: [75444224, 75448320) 75448320, 4096 00020000:00200000:2.0:1789760794.669864:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.669880:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18419, 18419] 00000008:00010000:2.0:1789760794.669896:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.669901:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.669910:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75444224, 75448320) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.669921:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75444224=0x47f3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.670114:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.670117:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75448320 00000020:00010000:2.0:1789760794.670124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.670127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.670130:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.670132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.670135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.670141:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.670147:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.670162:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760794.670255:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 75444224<>75448320 00000100:00100000:3.0:1789760794.670337:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565588864:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.684178:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985727 gen 3 00000008:00010000:2.0:1789760794.684213:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760794.684219:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.684227:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.684286:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760794.684307:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565588864:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760794.684378:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18419, 18419] 00000080:00200000:0.0:1789760794.684399:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760794.684404:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760794.684712:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75448320 i_blocks 147384 00000020:00200000:0.0:1789760794.684716:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760794.684726:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760794.684750:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75448320, count: 4096 00000080:00200000:0.0:1789760794.684757:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75448320, bytes: 4096 00000080:00200000:0.0:1789760794.684765:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760794.684772:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760794.684774:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75448320, 75452416) 0 0 00000080:00200000:0.0:1789760794.684778:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760794.684790:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18420, 18420] 00020000:00200000:0.0:1789760794.684797:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75448320 chunk: [75448320, 75452416) 75452416, 4096 00020000:00200000:0.0:1789760794.684801:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760794.684816:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18420, 18420] 00000008:00010000:0.0:1789760794.684828:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760794.684833:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760794.684841:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75448320, 75452416) append 0 append lockpos 0 00000080:00200000:0.0:1789760794.684851:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75448320=0x47f4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760794.684896:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760794.684899:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75452416 00000020:00010000:0.0:1789760794.684905:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760794.684908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.684911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.684914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760794.684917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760794.684923:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760794.684928:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760794.684944:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760794.685038:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75448320<>75452416 00000100:00100000:1.0:1789760794.685108:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565588992:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.712961:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985728 gen 3 00000008:00010000:0.0:1789760794.712998:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760794.713004:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.713013:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.713035:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760794.713057:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565588992:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760794.713151:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18420, 18420] 00000080:00200000:2.0:1789760794.713162:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760794.713167:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760794.713179:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75452416 i_blocks 147392 00000020:00200000:2.0:1789760794.713182:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760794.713191:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760794.713217:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75452416, count: 4096 00000080:00200000:2.0:1789760794.713223:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75452416, bytes: 4096 00000080:00200000:2.0:1789760794.713232:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760794.713239:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760794.713241:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75452416, 75456512) 0 0 00000080:00200000:2.0:1789760794.713245:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760794.713258:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18421, 18421] 00020000:00200000:2.0:1789760794.713264:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75452416 chunk: [75452416, 75456512) 75456512, 4096 00020000:00200000:2.0:1789760794.713268:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760794.713282:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18421, 18421] 00000008:00010000:2.0:1789760794.713296:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760794.713301:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760794.713309:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75452416, 75456512) append 0 append lockpos 0 00000080:00200000:2.0:1789760794.713320:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75452416=0x47f5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760794.713374:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760794.713376:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75456512 00000020:00010000:2.0:1789760794.713383:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760794.713397:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.713400:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760794.713403:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760794.713406:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760794.713412:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760794.713418:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760794.713432:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760794.713525:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 75452416<>75456512 00000100:00100000:2.0:1789760794.713563:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760794.713576:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565589120:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760794.730257:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985729 gen 3 00000008:00010000:2.0:1789760794.730296:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760794.730302:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760794.730312:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760794.730385:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760794.730425:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565589120:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760794.733016:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18421, 18421] 00000080:00200000:0.0:1789760794.733035:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760794.733040:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760794.733054:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75456512 i_blocks 147400 00000020:00200000:0.0:1789760794.733059:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760794.733071:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760794.733095:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75456512, count: 4096 00000080:00200000:0.0:1789760794.733103:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75456512, bytes: 4096 00000080:00200000:0.0:1789760794.733113:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760794.733121:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760794.733123:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75456512, 75460608) 0 0 00000080:00200000:0.0:1789760794.733127:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760794.733139:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18422, 18422] 00020000:00200000:0.0:1789760794.733145:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75456512 chunk: [75456512, 75460608) 75460608, 4096 00020000:00200000:0.0:1789760794.733149:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760794.733164:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18422, 18422] 00000008:00010000:0.0:1789760794.733178:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760794.733182:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760794.733191:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75456512, 75460608) append 0 append lockpos 0 00000080:00200000:0.0:1789760794.733202:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75456512=0x47f6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760794.733250:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760794.733253:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75460608 00000020:00010000:0.0:1789760794.733259:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760794.733262:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.733264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760794.733268:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760794.733270:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760794.733278:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760794.733283:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760794.733301:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760794.734161:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75456512<>75460608 00000100:00100000:0.0:1789760794.734207:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760794.734226:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565589248:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.755289:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985730 gen 3 00000008:00010000:0.0:1789760794.755325:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760794.755330:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.755338:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.755358:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760794.755379:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565589248:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.755848:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18422, 18422] 00000080:00200000:1.0:1789760794.755863:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.755869:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.755881:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75460608 i_blocks 147408 00000020:00200000:1.0:1789760794.755886:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.755895:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.755934:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75460608, count: 4096 00000080:00200000:1.0:1789760794.755941:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75460608, bytes: 4096 00000080:00200000:1.0:1789760794.755952:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.755960:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.755963:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75460608, 75464704) 0 0 00000080:00200000:1.0:1789760794.755967:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.755982:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18423, 18423] 00020000:00200000:1.0:1789760794.756000:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75460608 chunk: [75460608, 75464704) 75464704, 4096 00020000:00200000:1.0:1789760794.756004:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.756020:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18423, 18423] 00000008:00010000:1.0:1789760794.756035:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.756040:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.756049:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75460608, 75464704) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.756059:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75460608=0x47f7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.756100:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.756103:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75464704 00000020:00010000:1.0:1789760794.756109:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.756112:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.756115:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.756132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.756135:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.756141:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.756502:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.756521:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760794.756619:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 75460608<>75464704 00000100:00100000:1.0:1789760794.756682:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760794.756699:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565589376:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.782957:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985731 gen 3 00000008:00010000:0.0:1789760794.782997:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760794.783004:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.783012:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.783033:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760794.783060:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565589376:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.783155:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18423, 18423] 00000080:00200000:1.0:1789760794.783167:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.783172:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.783183:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75464704 i_blocks 147416 00000020:00200000:1.0:1789760794.783187:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.783195:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.783220:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75464704, count: 4096 00000080:00200000:1.0:1789760794.783227:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75464704, bytes: 4096 00000080:00200000:1.0:1789760794.783234:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.783241:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.783243:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75464704, 75468800) 0 0 00000080:00200000:1.0:1789760794.783247:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.783261:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18424, 18424] 00020000:00200000:1.0:1789760794.783269:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75464704 chunk: [75464704, 75468800) 75468800, 4096 00020000:00200000:1.0:1789760794.783273:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.783291:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18424, 18424] 00000008:00010000:1.0:1789760794.783305:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.783310:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.783319:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75464704, 75468800) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.783330:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75464704=0x47f8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.783468:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.783471:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75468800 00000020:00010000:1.0:1789760794.783478:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.783481:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.783484:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.783488:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.783491:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.783497:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.783504:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.783519:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760794.783629:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 75464704<>75468800 00000100:00100000:0.0:1789760794.784041:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760794.784058:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565589504:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.797257:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985732 gen 3 00000008:00010000:0.0:1789760794.797296:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760794.797303:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.797311:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.797329:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760794.797353:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565589504:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.797457:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18424, 18424] 00000080:00200000:1.0:1789760794.797469:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.797474:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.797485:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75468800 i_blocks 147424 00000020:00200000:1.0:1789760794.797488:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.797495:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.797522:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75468800, count: 4096 00000080:00200000:1.0:1789760794.797527:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75468800, bytes: 4096 00000080:00200000:1.0:1789760794.797535:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.797542:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.797544:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75468800, 75472896) 0 0 00000080:00200000:1.0:1789760794.797548:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.797560:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18425, 18425] 00020000:00200000:1.0:1789760794.797568:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75468800 chunk: [75468800, 75472896) 75472896, 4096 00020000:00200000:1.0:1789760794.797571:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.797586:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18425, 18425] 00000008:00010000:1.0:1789760794.797599:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.797604:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.797613:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75468800, 75472896) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.797624:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75468800=0x47f9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.797694:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.797697:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75472896 00000020:00010000:1.0:1789760794.797703:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.797706:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.797710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.797713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.797716:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.797722:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.797728:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.797742:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760794.797837:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 75468800<>75472896 00000100:00100000:0.0:1789760794.798004:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565589632:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.813416:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985733 gen 3 00000008:00010000:0.0:1789760794.813463:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760794.813471:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.813481:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.813493:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760794.813520:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565589632:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.813778:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18425, 18425] 00000080:00200000:1.0:1789760794.813790:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.813794:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.813805:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75472896 i_blocks 147432 00000020:00200000:1.0:1789760794.813809:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.813817:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.813845:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75472896, count: 4096 00000080:00200000:1.0:1789760794.813851:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75472896, bytes: 4096 00000080:00200000:1.0:1789760794.813862:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.813870:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.813872:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75472896, 75476992) 0 0 00000080:00200000:1.0:1789760794.813877:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.813891:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18426, 18426] 00020000:00200000:1.0:1789760794.813899:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75472896 chunk: [75472896, 75476992) 75476992, 4096 00020000:00200000:1.0:1789760794.813903:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.813918:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18426, 18426] 00000008:00010000:1.0:1789760794.813930:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.813935:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.813946:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75472896, 75476992) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.813956:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75472896=0x47fa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.814003:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.814006:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75476992 00000020:00010000:1.0:1789760794.814013:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.814017:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.814019:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.814023:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.814027:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.814033:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.814039:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.814055:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760794.814512:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 75472896<>75476992 00000100:00100000:0.0:1789760794.815317:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760794.815468:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565589760:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.837357:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985734 gen 3 00000008:00010000:0.0:1789760794.837411:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760794.837418:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.837426:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.837447:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760794.837466:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565589760:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.837598:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18426, 18426] 00000080:00200000:1.0:1789760794.837611:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.837616:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.837628:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75476992 i_blocks 147440 00000020:00200000:1.0:1789760794.837631:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.837638:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.837679:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75476992, count: 4096 00000080:00200000:1.0:1789760794.837686:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75476992, bytes: 4096 00000080:00200000:1.0:1789760794.837693:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.837700:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.837702:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75476992, 75481088) 0 0 00000080:00200000:1.0:1789760794.837706:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.837717:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18427, 18427] 00020000:00200000:1.0:1789760794.837724:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75476992 chunk: [75476992, 75481088) 75481088, 4096 00020000:00200000:1.0:1789760794.837728:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.837741:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18427, 18427] 00000008:00010000:1.0:1789760794.837754:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.837759:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.837767:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75476992, 75481088) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.837777:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75476992=0x47fb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.837818:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.837821:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75481088 00000020:00010000:1.0:1789760794.837827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.837830:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.837833:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.837836:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.837840:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.837845:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.837851:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.837864:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760794.837966:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 75476992<>75481088 00000100:00100000:0.0:1789760794.838190:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565589888:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.851593:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985735 gen 3 00000008:00010000:0.0:1789760794.851629:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760794.851635:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.851842:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.851861:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760794.851885:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565589888:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.852067:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18427, 18427] 00000080:00200000:1.0:1789760794.852080:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.852085:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.852096:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75481088 i_blocks 147448 00000020:00200000:1.0:1789760794.852099:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.852107:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.852130:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75481088, count: 4096 00000080:00200000:1.0:1789760794.852136:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75481088, bytes: 4096 00000080:00200000:1.0:1789760794.852143:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.852150:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.852153:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75481088, 75485184) 0 0 00000080:00200000:1.0:1789760794.852156:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.852169:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18428, 18428] 00020000:00200000:1.0:1789760794.852176:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75481088 chunk: [75481088, 75485184) 75485184, 4096 00020000:00200000:1.0:1789760794.852180:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.852194:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18428, 18428] 00000008:00010000:1.0:1789760794.852207:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.852212:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.852221:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75481088, 75485184) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.852231:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75481088=0x47fc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.852272:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.852274:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75485184 00000020:00010000:1.0:1789760794.852281:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.852284:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.852287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.852290:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.852293:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.852299:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.852304:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.852330:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760794.852437:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 75481088<>75485184 00000100:00100000:0.0:1789760794.852622:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760794.852638:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565590016:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.867719:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985736 gen 3 00000008:00010000:0.0:1789760794.867750:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760794.867756:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.867762:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.867780:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760794.867806:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565590016:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.867965:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18428, 18428] 00000080:00200000:1.0:1789760794.867977:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.867983:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.867995:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75485184 i_blocks 147456 00000020:00200000:1.0:1789760794.867998:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.868006:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.868031:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75485184, count: 4096 00000080:00200000:1.0:1789760794.868038:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75485184, bytes: 4096 00000080:00200000:1.0:1789760794.868045:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.868052:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.868054:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75485184, 75489280) 0 0 00000080:00200000:1.0:1789760794.868058:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.868072:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18429, 18429] 00020000:00200000:1.0:1789760794.868079:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75485184 chunk: [75485184, 75489280) 75489280, 4096 00020000:00200000:1.0:1789760794.868083:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.868098:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18429, 18429] 00000008:00010000:1.0:1789760794.868111:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.868116:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.868125:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75485184, 75489280) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.868135:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75485184=0x47fd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.868236:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.868239:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75489280 00000020:00010000:1.0:1789760794.868247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.868251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.868254:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.868257:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.868260:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.868266:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.868272:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.868285:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760794.868379:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 75485184<>75489280 00000100:00100000:0.0:1789760794.868696:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565590144:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.885284:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985737 gen 3 00000008:00010000:0.0:1789760794.885331:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760794.885339:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.885348:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.885370:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760794.885403:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565590144:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.885417:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18429, 18429] 00000080:00200000:1.0:1789760794.885431:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.885435:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.885447:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75489280 i_blocks 147464 00000020:00200000:1.0:1789760794.885450:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.885458:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.885485:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75489280, count: 4096 00000080:00200000:1.0:1789760794.885491:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75489280, bytes: 4096 00000080:00200000:1.0:1789760794.885502:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.885510:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.885512:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75489280, 75493376) 0 0 00000080:00200000:1.0:1789760794.885515:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.885528:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18430, 18430] 00020000:00200000:1.0:1789760794.885536:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75489280 chunk: [75489280, 75493376) 75493376, 4096 00020000:00200000:1.0:1789760794.885539:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.885553:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18430, 18430] 00000008:00010000:1.0:1789760794.885566:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.885571:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.885580:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75489280, 75493376) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.885591:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75489280=0x47fe000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.885635:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.885638:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75493376 00000020:00010000:1.0:1789760794.885676:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.885680:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.885683:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.885686:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.885689:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.885846:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.885851:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.885866:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760794.885960:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 75489280<>75493376 00000100:00100000:0.0:1789760794.886075:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565590272:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.897523:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985738 gen 3 00000008:00010000:0.0:1789760794.897562:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760794.897569:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.897577:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.897587:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760794.897611:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565590272:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.898013:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18430, 18430] 00000080:00200000:1.0:1789760794.898026:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.898043:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.898055:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75493376 i_blocks 147472 00000020:00200000:1.0:1789760794.898058:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.898077:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.898120:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75493376, count: 4096 00000080:00200000:1.0:1789760794.898126:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75493376, bytes: 4096 00000080:00200000:1.0:1789760794.898149:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.898166:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.898167:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75493376, 75497472) 0 0 00000080:00200000:1.0:1789760794.898171:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.898194:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18431, 18431] 00020000:00200000:1.0:1789760794.898201:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75493376 chunk: [75493376, 75497472) 75497472, 4096 00020000:00200000:1.0:1789760794.898205:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.898231:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18431, 18431] 00000008:00010000:1.0:1789760794.898244:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.898249:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.898270:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75493376, 75497472) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.898281:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75493376=0x47ff000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.898512:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.898516:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75497472 00000020:00010000:1.0:1789760794.898538:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.898542:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.898544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.898547:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.898551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.898557:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.899085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.899103:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760794.899201:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 75493376<>75497472 00000100:00100000:0.0:1789760794.899342:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565590400:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.910072:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985739 gen 3 00000008:00010000:0.0:1789760794.910109:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760794.910115:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.910124:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.910143:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760794.910164:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565590400:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.910251:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18431, 18431] 00000080:00200000:1.0:1789760794.910265:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.910269:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.910280:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75497472 i_blocks 147480 00000020:00200000:1.0:1789760794.910284:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.910291:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.910315:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75497472, count: 4096 00000080:00200000:1.0:1789760794.910550:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75497472, bytes: 4096 00000080:00200000:1.0:1789760794.910560:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.910566:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.910569:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75497472, 75501568) 0 0 00000080:00200000:1.0:1789760794.910572:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.910585:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18432, 18432] 00020000:00200000:1.0:1789760794.910593:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75497472 chunk: [75497472, 75501568) 75501568, 4096 00020000:00200000:1.0:1789760794.910596:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.910611:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18432, 18432] 00000008:00010000:1.0:1789760794.910624:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.910628:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.910637:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75497472, 75501568) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.910677:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75497472=0x4800000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.910720:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.910722:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75501568 00000020:00010000:1.0:1789760794.910729:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.910742:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.910745:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.910748:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.910751:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.910757:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.910762:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.910776:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760794.910865:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 75497472<>75501568 00000100:00100000:0.0:1789760794.911214:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760794.911230:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565590528:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.927693:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985740 gen 3 00000008:00010000:0.0:1789760794.927730:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760794.927746:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.927755:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.927772:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760794.927796:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565590528:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.927887:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18432, 18432] 00000080:00200000:1.0:1789760794.927899:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.927904:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.927916:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75501568 i_blocks 147488 00000020:00200000:1.0:1789760794.927919:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.927927:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.927951:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75501568, count: 4096 00000080:00200000:1.0:1789760794.927957:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75501568, bytes: 4096 00000080:00200000:1.0:1789760794.927964:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.927971:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.927973:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75501568, 75505664) 0 0 00000080:00200000:1.0:1789760794.927976:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.927990:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18433, 18433] 00020000:00200000:1.0:1789760794.927996:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75501568 chunk: [75501568, 75505664) 75505664, 4096 00020000:00200000:1.0:1789760794.928000:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.928015:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18433, 18433] 00000008:00010000:1.0:1789760794.928028:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.928032:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.928041:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75501568, 75505664) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.928050:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75501568=0x4801000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.928223:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.928225:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75505664 00000020:00010000:1.0:1789760794.928232:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.928236:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.928239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.928242:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.928245:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.928251:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.928262:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.928275:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760794.928370:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 75501568<>75505664 00000100:00100000:0.0:1789760794.928549:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565590656:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.947908:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985741 gen 3 00000008:00010000:0.0:1789760794.947946:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760794.947953:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.947961:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.947984:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760794.948007:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565590656:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.948625:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18433, 18433] 00000080:00200000:1.0:1789760794.948638:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.948846:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.948860:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75505664 i_blocks 147496 00000020:00200000:1.0:1789760794.948863:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.948872:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.948897:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75505664, count: 4096 00000080:00200000:1.0:1789760794.948903:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75505664, bytes: 4096 00000080:00200000:1.0:1789760794.948912:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.948920:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.948922:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75505664, 75509760) 0 0 00000080:00200000:1.0:1789760794.948926:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.948938:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18434, 18434] 00020000:00200000:1.0:1789760794.948944:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75505664 chunk: [75505664, 75509760) 75509760, 4096 00020000:00200000:1.0:1789760794.948948:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.948962:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18434, 18434] 00000008:00010000:1.0:1789760794.948973:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.948977:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.948985:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75505664, 75509760) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.948995:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75505664=0x4802000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.949036:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.949038:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75509760 00000020:00010000:1.0:1789760794.949044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.949047:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.949050:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.949053:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.949056:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.949061:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.949067:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.949081:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760794.949177:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 75505664<>75509760 00000100:00100000:0.0:1789760794.949433:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565590784:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.962707:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985742 gen 3 00000008:00010000:0.0:1789760794.962883:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760794.962890:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.962899:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.962914:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760794.962937:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565590784:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.963037:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18434, 18434] 00000080:00200000:1.0:1789760794.963050:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.963054:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.963067:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75509760 i_blocks 147504 00000020:00200000:1.0:1789760794.963070:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.963079:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.963104:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75509760, count: 4096 00000080:00200000:1.0:1789760794.963110:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75509760, bytes: 4096 00000080:00200000:1.0:1789760794.963117:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.963125:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.963127:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75509760, 75513856) 0 0 00000080:00200000:1.0:1789760794.963131:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.963143:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18435, 18435] 00020000:00200000:1.0:1789760794.963151:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75509760 chunk: [75509760, 75513856) 75513856, 4096 00020000:00200000:1.0:1789760794.963154:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.963168:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18435, 18435] 00000008:00010000:1.0:1789760794.963182:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.963187:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.963196:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75509760, 75513856) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.963207:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75509760=0x4803000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.963252:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.963254:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75513856 00000020:00010000:1.0:1789760794.963260:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.963263:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.963266:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.963269:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.963272:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.963278:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.963283:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.963297:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760794.963503:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 75509760<>75513856 00000100:00100000:0.0:1789760794.963627:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760794.963662:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565590912:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.981068:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985743 gen 3 00000008:00010000:0.0:1789760794.981107:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760794.981114:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.981123:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.981143:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760794.981168:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565590912:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.981421:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18435, 18435] 00000080:00200000:1.0:1789760794.981437:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.981442:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.981453:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75513856 i_blocks 147512 00000020:00200000:1.0:1789760794.981456:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.981465:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.981504:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75513856, count: 4096 00000080:00200000:1.0:1789760794.981511:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75513856, bytes: 4096 00000080:00200000:1.0:1789760794.981520:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.981527:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.981529:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75513856, 75517952) 0 0 00000080:00200000:1.0:1789760794.981532:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.981546:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18436, 18436] 00020000:00200000:1.0:1789760794.981552:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75513856 chunk: [75513856, 75517952) 75517952, 4096 00020000:00200000:1.0:1789760794.981556:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.981571:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18436, 18436] 00000008:00010000:1.0:1789760794.981586:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.981590:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.981600:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75513856, 75517952) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.981612:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75513856=0x4804000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.981695:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.981698:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75517952 00000020:00010000:1.0:1789760794.981705:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.981708:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.981711:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.981714:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.981717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.981723:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.981728:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.982831:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760794.982936:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 75513856<>75517952 00000100:00100000:0.0:1789760794.983004:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565591040:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760794.997881:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985744 gen 3 00000008:00010000:0.0:1789760794.997918:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760794.997925:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760794.997933:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760794.997952:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760794.997974:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565591040:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760794.998129:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18436, 18436] 00000080:00200000:1.0:1789760794.998143:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760794.998148:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760794.998159:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75517952 i_blocks 147520 00000020:00200000:1.0:1789760794.998163:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760794.998170:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760794.998196:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75517952, count: 4096 00000080:00200000:1.0:1789760794.998202:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75517952, bytes: 4096 00000080:00200000:1.0:1789760794.998210:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760794.998217:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760794.998219:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75517952, 75522048) 0 0 00000080:00200000:1.0:1789760794.998223:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760794.998235:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18437, 18437] 00020000:00200000:1.0:1789760794.998242:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75517952 chunk: [75517952, 75522048) 75522048, 4096 00020000:00200000:1.0:1789760794.998246:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760794.998262:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18437, 18437] 00000008:00010000:1.0:1789760794.998275:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760794.998279:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760794.998287:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75517952, 75522048) append 0 append lockpos 0 00000080:00200000:1.0:1789760794.998298:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75517952=0x4805000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760794.998351:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760794.998354:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75522048 00000020:00010000:1.0:1789760794.998362:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760794.998365:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.998368:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760794.998371:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760794.998375:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760794.998381:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760794.998397:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760794.998412:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760794.998500:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 75517952<>75522048 00000100:00100000:0.0:1789760794.998690:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565591168:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.010004:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985745 gen 3 00000008:00010000:0.0:1789760795.010038:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760795.010043:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.010052:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.010072:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760795.010096:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565591168:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.010233:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18437, 18437] 00000080:00200000:1.0:1789760795.010246:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.010251:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.010261:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75522048 i_blocks 147528 00000020:00200000:1.0:1789760795.010264:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.010271:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.010297:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75522048, count: 4096 00000080:00200000:1.0:1789760795.010303:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75522048, bytes: 4096 00000080:00200000:1.0:1789760795.010310:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.010317:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.010319:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75522048, 75526144) 0 0 00000080:00200000:1.0:1789760795.010322:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.010347:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18438, 18438] 00020000:00200000:1.0:1789760795.010355:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75522048 chunk: [75522048, 75526144) 75526144, 4096 00020000:00200000:1.0:1789760795.010358:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.010373:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18438, 18438] 00000008:00010000:1.0:1789760795.010397:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.010402:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.010411:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75522048, 75526144) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.010422:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75522048=0x4806000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.010465:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.010468:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75526144 00000020:00010000:1.0:1789760795.010475:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.010478:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.010481:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.010484:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.010488:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.010494:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.010500:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.010513:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760795.010603:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 75522048<>75526144 00000100:00100000:0.0:1789760795.010715:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760795.010739:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565591296:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.022004:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985746 gen 3 00000008:00010000:0.0:1789760795.022040:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760795.022047:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.022056:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.022076:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760795.022100:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565591296:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.022183:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18438, 18438] 00000080:00200000:1.0:1789760795.022195:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.022199:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.022210:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75526144 i_blocks 147536 00000020:00200000:1.0:1789760795.022213:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.022221:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.022244:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75526144, count: 4096 00000080:00200000:1.0:1789760795.022250:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75526144, bytes: 4096 00000080:00200000:1.0:1789760795.022257:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.022264:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.022266:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75526144, 75530240) 0 0 00000080:00200000:1.0:1789760795.022270:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.022283:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18439, 18439] 00020000:00200000:1.0:1789760795.022290:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75526144 chunk: [75526144, 75530240) 75530240, 4096 00020000:00200000:1.0:1789760795.022294:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.022308:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18439, 18439] 00000008:00010000:1.0:1789760795.022320:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.022335:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.022345:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75526144, 75530240) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.022355:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75526144=0x4807000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.022407:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.022410:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75530240 00000020:00010000:1.0:1789760795.022417:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.022421:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.022424:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.022427:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.022430:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.022437:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.022442:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.022455:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760795.022710:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75526144<>75530240 00000100:00100000:0.0:1789760795.022863:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565591424:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.035427:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985747 gen 3 00000008:00010000:0.0:1789760795.035466:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760795.035472:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.035481:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.035544:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760795.035569:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565591424:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.038275:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18439, 18439] 00000080:00200000:1.0:1789760795.038296:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.038301:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.038315:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75530240 i_blocks 147544 00000020:00200000:1.0:1789760795.038319:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.038331:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.038358:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75530240, count: 4096 00000080:00200000:1.0:1789760795.038364:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75530240, bytes: 4096 00000080:00200000:1.0:1789760795.038375:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.038393:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.038396:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75530240, 75534336) 0 0 00000080:00200000:1.0:1789760795.038399:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.038412:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18440, 18440] 00020000:00200000:1.0:1789760795.038419:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75530240 chunk: [75530240, 75534336) 75534336, 4096 00020000:00200000:1.0:1789760795.038423:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.038439:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18440, 18440] 00000008:00010000:1.0:1789760795.038452:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.038456:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.038466:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75530240, 75534336) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.038476:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75530240=0x4808000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.038528:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.038531:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75534336 00000020:00010000:1.0:1789760795.038539:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.038541:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.038544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.038547:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.038550:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.038557:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.038563:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.038586:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760795.038702:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 75530240<>75534336 00000100:00100000:0.0:1789760795.038772:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565591552:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.068768:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985748 gen 3 00000008:00010000:0.0:1789760795.068807:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760795.068815:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.068823:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.068842:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760795.068866:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565591552:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.069010:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18440, 18440] 00000080:00200000:1.0:1789760795.069023:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.069032:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.069044:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75534336 i_blocks 147552 00000020:00200000:1.0:1789760795.069047:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.069055:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.069081:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75534336, count: 4096 00000080:00200000:1.0:1789760795.069087:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75534336, bytes: 4096 00000080:00200000:1.0:1789760795.069095:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.069102:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.069104:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75534336, 75538432) 0 0 00000080:00200000:1.0:1789760795.069108:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.069120:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18441, 18441] 00020000:00200000:1.0:1789760795.069127:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75534336 chunk: [75534336, 75538432) 75538432, 4096 00020000:00200000:1.0:1789760795.069131:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.069145:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18441, 18441] 00000008:00010000:1.0:1789760795.069159:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.069164:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.069173:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75534336, 75538432) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.069182:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75534336=0x4809000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.069226:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.069228:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75538432 00000020:00010000:1.0:1789760795.069234:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.069237:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.069239:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.069242:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.069245:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.069250:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.069255:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.069269:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760795.069376:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 75534336<>75538432 00000100:00100000:0.0:1789760795.069447:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565591680:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.078079:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985749 gen 3 00000008:00010000:0.0:1789760795.078116:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760795.078123:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.078132:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.078150:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760795.078173:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565591680:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.078264:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18441, 18441] 00000080:00200000:1.0:1789760795.078289:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.078296:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.078308:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75538432 i_blocks 147560 00000020:00200000:1.0:1789760795.078311:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.078320:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.078353:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75538432, count: 4096 00000080:00200000:1.0:1789760795.078360:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75538432, bytes: 4096 00000080:00200000:1.0:1789760795.078368:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.078376:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.078378:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75538432, 75542528) 0 0 00000080:00200000:1.0:1789760795.078382:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.078406:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18442, 18442] 00020000:00200000:1.0:1789760795.078413:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75538432 chunk: [75538432, 75542528) 75542528, 4096 00020000:00200000:1.0:1789760795.078417:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.078431:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18442, 18442] 00000008:00010000:1.0:1789760795.078444:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.078449:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.078458:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75538432, 75542528) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.078469:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75538432=0x480a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.079804:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.079812:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75542528 00000020:00010000:1.0:1789760795.079823:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.079827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.079831:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.079834:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.079838:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.079847:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.079854:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.079878:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760795.079982:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 75538432<>75542528 00000100:00100000:0.0:1789760795.080055:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565591808:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.110099:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985750 gen 3 00000008:00010000:0.0:1789760795.110140:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760795.110147:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.110155:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.110174:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760795.110198:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565591808:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.110290:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18442, 18442] 00000080:00200000:1.0:1789760795.110303:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.110308:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.110319:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75542528 i_blocks 147568 00000020:00200000:1.0:1789760795.110328:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.110337:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.110361:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75542528, count: 4096 00000080:00200000:1.0:1789760795.110368:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75542528, bytes: 4096 00000080:00200000:1.0:1789760795.110375:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.110392:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.110395:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75542528, 75546624) 0 0 00000080:00200000:1.0:1789760795.110399:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.110414:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18443, 18443] 00020000:00200000:1.0:1789760795.110420:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75542528 chunk: [75542528, 75546624) 75546624, 4096 00020000:00200000:1.0:1789760795.110424:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.110440:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18443, 18443] 00000008:00010000:1.0:1789760795.110452:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.110457:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.110465:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75542528, 75546624) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.110475:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75542528=0x480b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.110691:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.110695:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75546624 00000020:00010000:1.0:1789760795.110702:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.110705:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.110708:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.110711:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.110713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.110720:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.110726:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.110752:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760795.110847:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 75542528<>75546624 00000100:00100000:0.0:1789760795.111003:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565591936:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.143185:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985751 gen 3 00000008:00010000:0.0:1789760795.143222:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760795.143228:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.143236:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.143255:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760795.143279:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565591936:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.143431:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18443, 18443] 00000080:00200000:1.0:1789760795.143444:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.143449:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.143460:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75546624 i_blocks 147576 00000020:00200000:1.0:1789760795.143464:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.143472:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.143499:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75546624, count: 4096 00000080:00200000:1.0:1789760795.143504:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75546624, bytes: 4096 00000080:00200000:1.0:1789760795.143512:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.143528:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.143530:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75546624, 75550720) 0 0 00000080:00200000:1.0:1789760795.143533:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.143545:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18444, 18444] 00020000:00200000:1.0:1789760795.143552:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75546624 chunk: [75546624, 75550720) 75550720, 4096 00020000:00200000:1.0:1789760795.143555:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.143570:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18444, 18444] 00000008:00010000:1.0:1789760795.143584:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.143588:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.143597:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75546624, 75550720) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.143607:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75546624=0x480c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.143684:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.143687:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75550720 00000020:00010000:1.0:1789760795.143694:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.143697:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.143699:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.143702:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.143705:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.143711:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.143717:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.143731:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760795.143829:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 75546624<>75550720 00000100:00100000:0.0:1789760795.144312:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760795.144341:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565592064:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.162833:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985752 gen 3 00000008:00010000:0.0:1789760795.162864:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760795.162868:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.162874:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.162889:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760795.162907:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565592064:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.162989:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18444, 18444] 00000080:00200000:1.0:1789760795.162999:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.163002:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.163011:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75550720 i_blocks 147584 00000020:00200000:1.0:1789760795.163013:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.163019:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.163038:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75550720, count: 4096 00000080:00200000:1.0:1789760795.163042:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75550720, bytes: 4096 00000080:00200000:1.0:1789760795.163048:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.163053:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.163055:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75550720, 75554816) 0 0 00000080:00200000:1.0:1789760795.163058:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.163067:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18445, 18445] 00020000:00200000:1.0:1789760795.163072:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75550720 chunk: [75550720, 75554816) 75554816, 4096 00020000:00200000:1.0:1789760795.163074:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.163086:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18445, 18445] 00000008:00010000:1.0:1789760795.163095:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.163098:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.163105:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75550720, 75554816) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.163115:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75550720=0x480d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.163156:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.163159:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75554816 00000020:00010000:1.0:1789760795.163165:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.163169:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.163171:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.163175:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.163184:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.163190:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.163195:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.163208:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760795.163295:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 75550720<>75554816 00010000:00010000:3.0:1789760795.163400:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760795.163443:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760795.163456:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565592192:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.185641:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985753 gen 3 00000008:00010000:0.0:1789760795.185694:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760795.185701:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.185708:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.185718:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760795.185747:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565592192:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.186049:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18445, 18445] 00000080:00200000:1.0:1789760795.186061:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.186065:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.186077:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75554816 i_blocks 147592 00000020:00200000:1.0:1789760795.186081:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.186088:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.186111:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75554816, count: 4096 00000080:00200000:1.0:1789760795.186117:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75554816, bytes: 4096 00000080:00200000:1.0:1789760795.186127:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.186134:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.186136:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75554816, 75558912) 0 0 00000080:00200000:1.0:1789760795.186140:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.186154:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18446, 18446] 00020000:00200000:1.0:1789760795.186161:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75554816 chunk: [75554816, 75558912) 75558912, 4096 00020000:00200000:1.0:1789760795.186165:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.186179:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18446, 18446] 00000008:00010000:1.0:1789760795.186194:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.186199:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.186208:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75554816, 75558912) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.186219:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75554816=0x480e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.186259:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.186261:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75558912 00000020:00010000:1.0:1789760795.186268:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.186271:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.186273:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.186276:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.186279:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.186285:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.186290:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.186304:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760795.186418:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 75554816<>75558912 00000100:00100000:0.0:1789760795.187676:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760795.187702:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565592320:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.199272:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985754 gen 3 00000008:00010000:0.0:1789760795.199315:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760795.199321:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.199340:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.199358:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760795.199380:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565592320:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.199493:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18446, 18446] 00000080:00200000:1.0:1789760795.199684:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.199689:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.199703:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75558912 i_blocks 147600 00000020:00200000:1.0:1789760795.199707:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.199715:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.199752:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75558912, count: 4096 00000080:00200000:1.0:1789760795.199759:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75558912, bytes: 4096 00000080:00200000:1.0:1789760795.199767:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.199774:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.199776:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75558912, 75563008) 0 0 00000080:00200000:1.0:1789760795.199780:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.199792:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18447, 18447] 00020000:00200000:1.0:1789760795.199799:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75558912 chunk: [75558912, 75563008) 75563008, 4096 00020000:00200000:1.0:1789760795.199802:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.199817:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18447, 18447] 00000008:00010000:1.0:1789760795.199829:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.199833:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.199841:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75558912, 75563008) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.199851:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75558912=0x480f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.199895:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.199897:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75563008 00000020:00010000:1.0:1789760795.199904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.199907:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.199910:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.199913:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.199916:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.199922:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.199928:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.199942:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760795.200031:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 75558912<>75563008 00000100:00100000:0.0:1789760795.200095:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565592448:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.218356:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985755 gen 3 00000008:00010000:0.0:1789760795.218401:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760795.218408:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.218418:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.218435:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760795.218459:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565592448:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.218546:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18447, 18447] 00000080:00200000:1.0:1789760795.218560:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.218564:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.218576:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75563008 i_blocks 147608 00000020:00200000:1.0:1789760795.218580:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.218588:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.218614:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75563008, count: 4096 00000080:00200000:1.0:1789760795.218620:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75563008, bytes: 4096 00000080:00200000:1.0:1789760795.218628:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.218636:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.218638:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75563008, 75567104) 0 0 00000080:00200000:1.0:1789760795.218642:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.218680:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18448, 18448] 00020000:00200000:1.0:1789760795.218687:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75563008 chunk: [75563008, 75567104) 75567104, 4096 00020000:00200000:1.0:1789760795.218690:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.218707:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18448, 18448] 00000008:00010000:1.0:1789760795.218721:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.218738:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.218749:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75563008, 75567104) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.218759:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75563008=0x4810000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.218803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.218806:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75567104 00000020:00010000:1.0:1789760795.218813:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.218816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.218820:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.218823:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.218826:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.218833:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.218839:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.218854:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760795.218953:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 75563008<>75567104 00000100:00100000:0.0:1789760795.219179:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565592576:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.230249:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985756 gen 3 00000008:00010000:0.0:1789760795.230290:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760795.230296:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.230304:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.230323:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760795.230343:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565592576:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.230470:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18448, 18448] 00000080:00200000:1.0:1789760795.230482:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.230486:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.230497:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75567104 i_blocks 147616 00000020:00200000:1.0:1789760795.230501:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.230508:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.230535:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75567104, count: 4096 00000080:00200000:1.0:1789760795.230540:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75567104, bytes: 4096 00000080:00200000:1.0:1789760795.230548:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.230555:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.230557:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75567104, 75571200) 0 0 00000080:00200000:1.0:1789760795.230561:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.230572:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18449, 18449] 00020000:00200000:1.0:1789760795.230578:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75567104 chunk: [75567104, 75571200) 75571200, 4096 00020000:00200000:1.0:1789760795.230581:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.230595:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18449, 18449] 00000008:00010000:1.0:1789760795.230606:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.230610:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.230618:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75567104, 75571200) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.230628:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75567104=0x4811000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.230688:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.230691:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75571200 00000020:00010000:1.0:1789760795.230697:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.230700:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.230703:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.230706:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.230710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.230716:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.230721:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.230734:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760795.230818:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 75567104<>75571200 00000100:00100000:0.0:1789760795.231357:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760795.231373:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565592704:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.244162:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985757 gen 3 00000008:00010000:0.0:1789760795.244198:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760795.244205:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.244213:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.244231:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760795.244254:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565592704:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.244402:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18449, 18449] 00000080:00200000:1.0:1789760795.244418:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.244423:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.244435:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75571200 i_blocks 147624 00000020:00200000:1.0:1789760795.244439:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.244447:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.244471:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75571200, count: 4096 00000080:00200000:1.0:1789760795.244476:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75571200, bytes: 4096 00000080:00200000:1.0:1789760795.244494:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.244502:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.244504:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75571200, 75575296) 0 0 00000080:00200000:1.0:1789760795.244507:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.244529:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18450, 18450] 00020000:00200000:1.0:1789760795.244536:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75571200 chunk: [75571200, 75575296) 75575296, 4096 00020000:00200000:1.0:1789760795.244540:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.244556:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18450, 18450] 00000008:00010000:1.0:1789760795.244569:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.244574:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.244583:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75571200, 75575296) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.244593:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75571200=0x4812000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.244638:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.244640:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75575296 00000020:00010000:1.0:1789760795.244681:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.244684:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.244687:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.244690:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.244693:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.244699:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.244704:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.244717:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760795.244823:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 75571200<>75575296 00000100:00100000:0.0:1789760795.245636:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565592832:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.258616:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985758 gen 3 00000008:00010000:0.0:1789760795.258676:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760795.258682:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.258690:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.258707:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760795.258729:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565592832:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.258817:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18450, 18450] 00000080:00200000:1.0:1789760795.258831:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.258835:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.258847:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75575296 i_blocks 147632 00000020:00200000:1.0:1789760795.258850:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.258858:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.258883:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75575296, count: 4096 00000080:00200000:1.0:1789760795.258890:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75575296, bytes: 4096 00000080:00200000:1.0:1789760795.258897:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.258904:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.258907:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75575296, 75579392) 0 0 00000080:00200000:1.0:1789760795.258910:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.258922:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18451, 18451] 00020000:00200000:1.0:1789760795.258927:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75575296 chunk: [75575296, 75579392) 75579392, 4096 00020000:00200000:1.0:1789760795.258930:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.258942:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18451, 18451] 00000008:00010000:1.0:1789760795.258952:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.258956:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.258963:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75575296, 75579392) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.258971:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75575296=0x4813000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.259005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.259007:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75579392 00000020:00010000:1.0:1789760795.259012:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.259014:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.259016:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.259017:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.259020:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.259023:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.259027:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.259037:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760795.259123:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 75575296<>75579392 00000100:00100000:0.0:1789760795.259199:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760795.259209:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565592960:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.267602:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985759 gen 3 00000008:00010000:0.0:1789760795.267638:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760795.267674:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.267684:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.267738:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760795.267756:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565592960:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.267847:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18451, 18451] 00000080:00200000:1.0:1789760795.267858:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.267861:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.267871:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75579392 i_blocks 147640 00000020:00200000:1.0:1789760795.267873:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.267879:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.267900:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75579392, count: 4096 00000080:00200000:1.0:1789760795.267905:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75579392, bytes: 4096 00000080:00200000:1.0:1789760795.267910:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.267916:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.267918:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75579392, 75583488) 0 0 00000080:00200000:1.0:1789760795.267920:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.267930:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18452, 18452] 00020000:00200000:1.0:1789760795.267934:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75579392 chunk: [75579392, 75583488) 75583488, 4096 00020000:00200000:1.0:1789760795.267937:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.267949:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18452, 18452] 00000008:00010000:1.0:1789760795.267958:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.267961:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.267968:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75579392, 75583488) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.267976:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75579392=0x4814000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.268068:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.268070:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75583488 00000020:00010000:1.0:1789760795.268078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.268081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.268084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.268086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.268089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.268095:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.268100:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.268113:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760795.268204:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 75579392<>75583488 00000100:00100000:0.0:1789760795.268258:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565593088:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.277244:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985760 gen 3 00000008:00010000:0.0:1789760795.277281:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760795.277288:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.277296:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.277315:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760795.277338:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565593088:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.277452:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18452, 18452] 00000080:00200000:1.0:1789760795.277466:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.277471:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.277704:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75583488 i_blocks 147648 00000020:00200000:1.0:1789760795.277709:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.277717:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.277751:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75583488, count: 4096 00000080:00200000:1.0:1789760795.277758:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75583488, bytes: 4096 00000080:00200000:1.0:1789760795.277764:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.277771:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.277772:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75583488, 75587584) 0 0 00000080:00200000:1.0:1789760795.277776:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.277787:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18453, 18453] 00020000:00200000:1.0:1789760795.277794:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75583488 chunk: [75583488, 75587584) 75587584, 4096 00020000:00200000:1.0:1789760795.277797:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.277810:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18453, 18453] 00000008:00010000:1.0:1789760795.277821:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.277825:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.277833:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75583488, 75587584) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.277843:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75583488=0x4815000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.277879:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.277881:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75587584 00000020:00010000:1.0:1789760795.277887:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.277890:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.277892:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.277895:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.277898:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.277903:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.277908:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.277920:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760795.278008:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 75583488<>75587584 00000100:00100000:0.0:1789760795.278081:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565593216:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.288498:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985761 gen 3 00000008:00010000:0.0:1789760795.288597:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760795.288602:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.288609:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.288676:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760795.288695:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565593216:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.288807:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18453, 18453] 00000080:00200000:1.0:1789760795.288818:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.288821:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.288831:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75587584 i_blocks 147656 00000020:00200000:1.0:1789760795.288833:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.288839:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.288860:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75587584, count: 4096 00000080:00200000:1.0:1789760795.288864:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75587584, bytes: 4096 00000080:00200000:1.0:1789760795.288869:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.288875:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.288877:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75587584, 75591680) 0 0 00000080:00200000:1.0:1789760795.288879:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.288889:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18454, 18454] 00020000:00200000:1.0:1789760795.288894:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75587584 chunk: [75587584, 75591680) 75591680, 4096 00020000:00200000:1.0:1789760795.288896:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.288908:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18454, 18454] 00000008:00010000:1.0:1789760795.288917:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.288920:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.288927:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75587584, 75591680) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.288934:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75587584=0x4816000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.288972:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.288975:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75591680 00000020:00010000:1.0:1789760795.288983:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.288986:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.288988:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.288991:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.288994:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.289000:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.289005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.289019:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760795.289112:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 75587584<>75591680 00000100:00100000:0.0:1789760795.289277:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565593344:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.312117:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985762 gen 3 00000008:00010000:0.0:1789760795.312151:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760795.312158:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.312166:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.312185:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760795.312206:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565593344:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.312537:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18454, 18454] 00000080:00200000:1.0:1789760795.312549:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.312554:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.312565:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75591680 i_blocks 147664 00000020:00200000:1.0:1789760795.312569:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.312576:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.312601:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75591680, count: 4096 00000080:00200000:1.0:1789760795.312607:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75591680, bytes: 4096 00000080:00200000:1.0:1789760795.312614:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.312621:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.312623:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75591680, 75595776) 0 0 00000080:00200000:1.0:1789760795.312626:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.312639:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18455, 18455] 00020000:00200000:1.0:1789760795.312668:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75591680 chunk: [75591680, 75595776) 75595776, 4096 00020000:00200000:1.0:1789760795.312672:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.312688:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18455, 18455] 00000008:00010000:1.0:1789760795.312699:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.312703:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.312712:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75591680, 75595776) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.312722:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75591680=0x4817000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.312763:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.312766:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75595776 00000020:00010000:1.0:1789760795.312772:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.312776:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.312778:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.312780:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.312783:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.312789:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.312794:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.312807:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760795.312897:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 75591680<>75595776 00000100:00100000:0.0:1789760795.313126:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565593472:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.332700:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985763 gen 3 00000008:00010000:0.0:1789760795.332734:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760795.332740:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.332749:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.332766:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760795.332788:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565593472:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.332890:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18455, 18455] 00000080:00200000:1.0:1789760795.332933:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.332938:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.332951:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75595776 i_blocks 147672 00000020:00200000:1.0:1789760795.332954:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.332961:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.332985:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75595776, count: 4096 00000080:00200000:1.0:1789760795.332990:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75595776, bytes: 4096 00000080:00200000:1.0:1789760795.332997:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.333004:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.333007:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75595776, 75599872) 0 0 00000080:00200000:1.0:1789760795.333010:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.333021:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18456, 18456] 00020000:00200000:1.0:1789760795.333027:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75595776 chunk: [75595776, 75599872) 75599872, 4096 00020000:00200000:1.0:1789760795.333031:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.333046:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18456, 18456] 00000008:00010000:1.0:1789760795.333059:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.333064:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.333073:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75595776, 75599872) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.333084:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75595776=0x4818000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.333201:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.333204:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75599872 00000020:00010000:1.0:1789760795.333214:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.333218:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.333221:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.333223:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.333226:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.333232:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.333240:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.333253:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760795.333422:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75595776<>75599872 00000100:00100000:0.0:1789760795.333521:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760795.333536:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565593600:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.345990:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985764 gen 3 00000008:00010000:0.0:1789760795.346024:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760795.346030:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.346039:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.346058:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760795.346080:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565593600:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.346231:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18456, 18456] 00000080:00200000:1.0:1789760795.346244:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.346248:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.346259:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75599872 i_blocks 147680 00000020:00200000:1.0:1789760795.346262:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.346268:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.346293:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75599872, count: 4096 00000080:00200000:1.0:1789760795.346299:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75599872, bytes: 4096 00000080:00200000:1.0:1789760795.346306:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.346314:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.346316:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75599872, 75603968) 0 0 00000080:00200000:1.0:1789760795.346393:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.346409:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18457, 18457] 00020000:00200000:1.0:1789760795.346416:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75599872 chunk: [75599872, 75603968) 75603968, 4096 00020000:00200000:1.0:1789760795.346420:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.346436:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18457, 18457] 00000008:00010000:1.0:1789760795.346449:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.346454:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.346462:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75599872, 75603968) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.346472:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75599872=0x4819000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.346516:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.346519:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75603968 00000020:00010000:1.0:1789760795.346526:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.346529:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.346532:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.346535:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.346538:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.346544:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.346549:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.346562:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760795.346774:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 75599872<>75603968 00000100:00100000:0.0:1789760795.346832:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565593728:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.362025:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985765 gen 3 00000008:00010000:0.0:1789760795.362060:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760795.362067:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.362075:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.362098:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760795.362131:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565593728:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760795.362292:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18457, 18457] 00000080:00200000:2.0:1789760795.362306:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760795.362311:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760795.362333:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75603968 i_blocks 147688 00000020:00200000:2.0:1789760795.362337:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760795.362346:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760795.362371:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75603968, count: 4096 00000080:00200000:2.0:1789760795.362376:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75603968, bytes: 4096 00000080:00200000:2.0:1789760795.362393:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760795.362399:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760795.362401:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75603968, 75608064) 0 0 00000080:00200000:2.0:1789760795.362405:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760795.362418:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18458, 18458] 00020000:00200000:2.0:1789760795.362433:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75603968 chunk: [75603968, 75608064) 75608064, 4096 00020000:00200000:2.0:1789760795.362438:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760795.362453:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18458, 18458] 00000008:00010000:2.0:1789760795.362468:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760795.362473:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760795.362482:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75603968, 75608064) append 0 append lockpos 0 00000080:00200000:2.0:1789760795.362503:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75603968=0x481a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760795.362550:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760795.362552:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75608064 00000020:00010000:2.0:1789760795.362559:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760795.362562:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.362565:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.362570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760795.362573:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760795.362579:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760795.362585:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760795.362600:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760795.362748:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 75603968<>75608064 00000100:00100000:3.0:1789760795.363103:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760795.363130:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565593856:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760795.382086:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985766 gen 3 00000008:00010000:2.0:1789760795.382195:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760795.382202:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760795.382212:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760795.382271:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760795.382295:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565593856:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760795.382435:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18458, 18458] 00000080:00200000:0.0:1789760795.382447:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760795.382451:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760795.382462:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75608064 i_blocks 147696 00000020:00200000:0.0:1789760795.382465:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760795.382472:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760795.382631:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75608064, count: 4096 00000080:00200000:0.0:1789760795.382639:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75608064, bytes: 4096 00000080:00200000:0.0:1789760795.382666:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760795.382673:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760795.382675:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75608064, 75612160) 0 0 00000080:00200000:0.0:1789760795.382679:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760795.382693:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18459, 18459] 00020000:00200000:0.0:1789760795.382700:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75608064 chunk: [75608064, 75612160) 75612160, 4096 00020000:00200000:0.0:1789760795.382704:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760795.382838:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18459, 18459] 00000008:00010000:0.0:1789760795.382894:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760795.382900:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760795.382911:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75608064, 75612160) append 0 append lockpos 0 00000080:00200000:0.0:1789760795.382922:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75608064=0x481b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760795.382967:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760795.382970:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75612160 00000020:00010000:0.0:1789760795.382976:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:0.0:1789760795.382979:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760795.382981:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760795.382985:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760795.382987:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760795.382993:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760795.382998:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760795.383013:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760795.383111:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75608064<>75612160 00000100:00100000:0.0:1789760795.383152:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760795.383167:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565593984:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.394554:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985767 gen 3 00000008:00010000:0.0:1789760795.394594:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:0.0:1789760795.394601:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.394611:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.394634:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:0.0:1789760795.394688:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565593984:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.395244:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18459, 18459] 00000080:00200000:1.0:1789760795.395257:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.395262:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.395274:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75612160 i_blocks 147704 00000020:00200000:1.0:1789760795.395277:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.395287:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.395313:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75612160, count: 4096 00000080:00200000:1.0:1789760795.395320:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75612160, bytes: 4096 00000080:00200000:1.0:1789760795.395330:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.395336:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.395339:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75612160, 75616256) 0 0 00000080:00200000:1.0:1789760795.395343:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.395359:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18460, 18460] 00020000:00200000:1.0:1789760795.395367:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75612160 chunk: [75612160, 75616256) 75616256, 4096 00020000:00200000:1.0:1789760795.395371:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.395399:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18460, 18460] 00000008:00010000:1.0:1789760795.395419:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.395435:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.395446:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75612160, 75616256) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.395460:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75612160=0x481c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.395534:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.395537:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75616256 00000020:00010000:1.0:1789760795.395544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.395548:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.395551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.395554:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.395558:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.395566:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.395573:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.395591:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760795.395754:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 75612160<>75616256 00000100:00100000:1.0:1789760795.395798:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760795.395817:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565594112:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.410433:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985768 gen 3 00000008:00010000:0.0:1789760795.410470:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760795.410477:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.410484:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.410514:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760795.410539:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565594112:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760795.411033:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18460, 18460] 00000080:00200000:2.0:1789760795.411045:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760795.411050:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760795.411062:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75616256 i_blocks 147712 00000020:00200000:2.0:1789760795.411065:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760795.411073:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760795.411099:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75616256, count: 4096 00000080:00200000:2.0:1789760795.411105:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75616256, bytes: 4096 00000080:00200000:2.0:1789760795.411112:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760795.411120:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760795.411122:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75616256, 75620352) 0 0 00000080:00200000:2.0:1789760795.411126:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760795.411138:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18461, 18461] 00020000:00200000:2.0:1789760795.411144:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75616256 chunk: [75616256, 75620352) 75620352, 4096 00020000:00200000:2.0:1789760795.411148:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760795.411163:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18461, 18461] 00000008:00010000:2.0:1789760795.411176:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760795.411181:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760795.411189:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75616256, 75620352) append 0 append lockpos 0 00000080:00200000:2.0:1789760795.411199:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75616256=0x481d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760795.411244:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760795.411246:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75620352 00000020:00010000:2.0:1789760795.411253:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_release():245 00000020:00010000:2.0:1789760795.411256:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.411258:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e638/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.411261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e8a8 (ffff93f84354e5a8/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760795.411263:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e5a8/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760795.411269:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760795.411275:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760795.411288:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760795.412060:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 75616256<>75620352 00000100:00100000:2.0:1789760795.412107:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565594240:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760795.425162:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985769 gen 3 00000008:00010000:2.0:1789760795.425200:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760795.425207:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760795.425215:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760795.425293:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760795.425319:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565594240:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760795.425497:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18461, 18461] 00000080:00200000:3.0:1789760795.425599:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760795.425605:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760795.425616:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75620352 i_blocks 147720 00000020:00200000:3.0:1789760795.425619:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760795.425625:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760795.425725:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75620352, count: 4096 00000080:00200000:3.0:1789760795.425733:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75620352, bytes: 4096 00000080:00200000:3.0:1789760795.425743:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760795.425749:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760795.425751:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75620352, 75624448) 0 0 00000080:00200000:3.0:1789760795.425755:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760795.425768:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18462, 18462] 00020000:00200000:3.0:1789760795.425775:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75620352 chunk: [75620352, 75624448) 75624448, 4096 00020000:00200000:3.0:1789760795.425779:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760795.425794:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18462, 18462] 00000008:00010000:3.0:1789760795.425807:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760795.425812:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760795.425822:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75620352, 75624448) append 0 append lockpos 0 00000080:00200000:3.0:1789760795.425832:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75620352=0x481e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760795.425876:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760795.425879:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75624448 00000020:00010000:3.0:1789760795.425885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:3.0:1789760795.425889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760795.425891:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760795.425895:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760795.425897:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760795.425904:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760795.425910:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760795.425923:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760795.426678:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d67100 - object 0x240000400:5 offset 75620352<>75624448 00000100:00100000:2.0:1789760795.426750:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565594368:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760795.446801:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985770 gen 3 00000008:00010000:2.0:1789760795.446841:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760795.446849:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760795.446859:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760795.446880:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760795.446904:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565594368:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.447003:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18462, 18462] 00000080:00200000:1.0:1789760795.447017:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.447022:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.447034:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75624448 i_blocks 147728 00000020:00200000:1.0:1789760795.447037:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.447046:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.447072:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75624448, count: 4096 00000080:00200000:1.0:1789760795.447079:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75624448, bytes: 4096 00000080:00200000:1.0:1789760795.447087:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.447094:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.447097:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75624448, 75628544) 0 0 00000080:00200000:1.0:1789760795.447101:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.447116:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18463, 18463] 00020000:00200000:1.0:1789760795.447123:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75624448 chunk: [75624448, 75628544) 75628544, 4096 00020000:00200000:1.0:1789760795.447127:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.447141:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18463, 18463] 00000008:00010000:1.0:1789760795.447154:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.447159:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.447168:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75624448, 75628544) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.447178:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75624448=0x481f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.447234:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.447237:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75628544 00000020:00010000:1.0:1789760795.447244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.447248:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.447251:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.447255:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.447257:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.447264:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.447269:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.447283:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760795.447377:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 75624448<>75628544 00000100:00100000:0.0:1789760795.447490:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565594496:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.463496:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985771 gen 3 00000008:00010000:0.0:1789760795.463542:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760795.463549:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.463557:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.463576:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000080:00200000:1.0:1789760795.463597:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18463, 18463] 00000100:00100000:0.0:1789760795.463600:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565594496:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.463610:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.463615:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.463626:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75628544 i_blocks 147736 00000020:00200000:1.0:1789760795.463629:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.463637:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.463686:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75628544, count: 4096 00000080:00200000:1.0:1789760795.463693:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75628544, bytes: 4096 00000080:00200000:1.0:1789760795.463700:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.463706:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.463709:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75628544, 75632640) 0 0 00000080:00200000:1.0:1789760795.463712:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.463726:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18464, 18464] 00020000:00200000:1.0:1789760795.463732:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75628544 chunk: [75628544, 75632640) 75632640, 4096 00020000:00200000:1.0:1789760795.463737:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.463753:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18464, 18464] 00000008:00010000:1.0:1789760795.463766:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.463771:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.463779:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75628544, 75632640) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.463790:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75628544=0x4820000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.463834:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.463836:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75632640 00000020:00010000:1.0:1789760795.463843:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.463846:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.463849:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.463852:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.463855:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.463860:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.463865:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.463878:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760795.463972:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 75628544<>75632640 00000100:00100000:0.0:1789760795.464038:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565594624:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.482250:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985772 gen 3 00000008:00010000:0.0:1789760795.482287:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760795.482293:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.482300:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.482324:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760795.482348:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565594624:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.482639:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18464, 18464] 00000080:00200000:1.0:1789760795.482703:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.482709:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.482721:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75632640 i_blocks 147744 00000020:00200000:1.0:1789760795.482725:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.482742:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.482767:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75632640, count: 4096 00000080:00200000:1.0:1789760795.482773:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75632640, bytes: 4096 00000080:00200000:1.0:1789760795.482781:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.482789:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.482791:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75632640, 75636736) 0 0 00000080:00200000:1.0:1789760795.482794:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.482806:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18465, 18465] 00020000:00200000:1.0:1789760795.482813:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75632640 chunk: [75632640, 75636736) 75636736, 4096 00020000:00200000:1.0:1789760795.482816:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.482829:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18465, 18465] 00000008:00010000:1.0:1789760795.482842:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.482846:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.482854:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75632640, 75636736) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.482864:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75632640=0x4821000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.482904:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.482906:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75636736 00000020:00010000:1.0:1789760795.482912:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.482915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.482918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.482921:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.482924:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.482930:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.482935:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.482949:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760795.483043:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 75632640<>75636736 00000100:00100000:1.0:1789760795.483082:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565594752:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.496391:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985773 gen 3 00000008:00010000:0.0:1789760795.496436:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760795.496443:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.496451:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.496469:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760795.496491:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565594752:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.496670:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18465, 18465] 00000080:00200000:1.0:1789760795.496684:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.496689:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.496700:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75636736 i_blocks 147752 00000020:00200000:1.0:1789760795.496703:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.496711:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.496735:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75636736, count: 4096 00000080:00200000:1.0:1789760795.496741:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75636736, bytes: 4096 00000080:00200000:1.0:1789760795.496748:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.496755:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.496757:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75636736, 75640832) 0 0 00000080:00200000:1.0:1789760795.496760:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.496772:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18466, 18466] 00020000:00200000:1.0:1789760795.496779:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75636736 chunk: [75636736, 75640832) 75640832, 4096 00020000:00200000:1.0:1789760795.496782:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.496796:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18466, 18466] 00000008:00010000:1.0:1789760795.496809:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.496813:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.496821:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75636736, 75640832) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.496832:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75636736=0x4822000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.496875:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.496878:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75640832 00000020:00010000:1.0:1789760795.496885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.496888:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.496890:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.496893:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.496895:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.496901:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.496907:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.496919:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760795.497012:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 75636736<>75640832 00000100:00100000:0.0:1789760795.497108:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760795.497123:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565594880:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.526422:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985774 gen 3 00000008:00010000:0.0:1789760795.526457:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760795.526462:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.526470:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.526488:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760795.526509:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565594880:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.526757:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18466, 18466] 00000080:00200000:1.0:1789760795.526769:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.526774:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.526784:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75640832 i_blocks 147760 00000020:00200000:1.0:1789760795.526788:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.526795:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.526818:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75640832, count: 4096 00000080:00200000:1.0:1789760795.526823:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75640832, bytes: 4096 00000080:00200000:1.0:1789760795.526830:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.526838:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.526840:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75640832, 75644928) 0 0 00000080:00200000:1.0:1789760795.526844:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.526856:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18467, 18467] 00020000:00200000:1.0:1789760795.526862:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75640832 chunk: [75640832, 75644928) 75644928, 4096 00020000:00200000:1.0:1789760795.526866:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.526879:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18467, 18467] 00000008:00010000:1.0:1789760795.526892:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.526897:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.526906:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75640832, 75644928) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.526916:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75640832=0x4823000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.526958:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.526961:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75644928 00000020:00010000:1.0:1789760795.526968:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.526971:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.527157:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.527162:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.527164:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.527171:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.527177:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.527190:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760795.527280:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 75640832<>75644928 00000100:00100000:0.0:1789760795.527340:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565595008:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.538568:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985775 gen 3 00000008:00010000:0.0:1789760795.538603:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760795.538610:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.538618:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.538638:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760795.538684:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565595008:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.538766:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18467, 18467] 00000080:00200000:1.0:1789760795.538780:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.538784:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.538796:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75644928 i_blocks 147768 00000020:00200000:1.0:1789760795.538799:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.538807:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.538841:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75644928, count: 4096 00000080:00200000:1.0:1789760795.538847:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75644928, bytes: 4096 00000080:00200000:1.0:1789760795.538855:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.538862:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.538864:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75644928, 75649024) 0 0 00000080:00200000:1.0:1789760795.538868:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.538882:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18468, 18468] 00020000:00200000:1.0:1789760795.538889:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75644928 chunk: [75644928, 75649024) 75649024, 4096 00020000:00200000:1.0:1789760795.538893:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.538908:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18468, 18468] 00000008:00010000:1.0:1789760795.538921:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.538927:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.538935:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75644928, 75649024) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.538946:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75644928=0x4824000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.538990:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.538993:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75649024 00000020:00010000:1.0:1789760795.539000:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.539004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.539007:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.539010:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.539013:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.539018:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.539024:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.539038:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760795.539128:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 75644928<>75649024 00000100:00100000:0.0:1789760795.539277:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565595136:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.551449:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985776 gen 3 00000008:00010000:0.0:1789760795.551532:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760795.551538:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.551547:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.551625:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760795.551677:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565595136:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.552195:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18468, 18468] 00000080:00200000:1.0:1789760795.552209:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.552214:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.552228:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75649024 i_blocks 147776 00000020:00200000:1.0:1789760795.552233:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.552241:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.552267:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75649024, count: 4096 00000080:00200000:1.0:1789760795.552273:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75649024, bytes: 4096 00000080:00200000:1.0:1789760795.552283:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.552290:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.552292:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75649024, 75653120) 0 0 00000080:00200000:1.0:1789760795.552296:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.552309:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18469, 18469] 00020000:00200000:1.0:1789760795.552316:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75649024 chunk: [75649024, 75653120) 75653120, 4096 00020000:00200000:1.0:1789760795.552407:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.552424:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18469, 18469] 00000008:00010000:1.0:1789760795.552437:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.552442:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.552451:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75649024, 75653120) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.552463:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75649024=0x4825000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.552575:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.552578:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75653120 00000020:00010000:1.0:1789760795.552587:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.552590:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.552594:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.552598:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.552600:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.552607:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.552613:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.552632:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760795.552908:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 75649024<>75653120 00000100:00100000:0.0:1789760795.552977:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565595264:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.565939:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985777 gen 3 00000008:00010000:0.0:1789760795.565976:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760795.565983:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.565991:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.566010:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760795.566034:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565595264:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.566123:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18469, 18469] 00000080:00200000:1.0:1789760795.566136:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.566141:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.566154:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75653120 i_blocks 147784 00000020:00200000:1.0:1789760795.566158:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.566166:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.566193:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75653120, count: 4096 00000080:00200000:1.0:1789760795.566198:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75653120, bytes: 4096 00000080:00200000:1.0:1789760795.566206:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.566213:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.566215:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75653120, 75657216) 0 0 00000080:00200000:1.0:1789760795.566219:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.566232:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18470, 18470] 00020000:00200000:1.0:1789760795.566239:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75653120 chunk: [75653120, 75657216) 75657216, 4096 00020000:00200000:1.0:1789760795.566243:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.566258:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18470, 18470] 00000008:00010000:1.0:1789760795.566271:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.566276:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.566284:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75653120, 75657216) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.566294:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75653120=0x4826000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.566413:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.566416:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75657216 00000020:00010000:1.0:1789760795.566424:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.566428:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.566431:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.566434:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.566437:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.566442:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.566448:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.566462:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760795.566557:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 75653120<>75657216 00000100:00100000:0.0:1789760795.566616:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565595392:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.578488:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985778 gen 3 00000008:00010000:0.0:1789760795.578524:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760795.578530:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.578538:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.578601:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760795.578624:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565595392:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.578826:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18470, 18470] 00000080:00200000:1.0:1789760795.578840:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.578845:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.578857:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75657216 i_blocks 147792 00000020:00200000:1.0:1789760795.578860:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.578868:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.578895:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75657216, count: 4096 00000080:00200000:1.0:1789760795.578902:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75657216, bytes: 4096 00000080:00200000:1.0:1789760795.578910:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.578916:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.578918:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75657216, 75661312) 0 0 00000080:00200000:1.0:1789760795.578922:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.578935:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18471, 18471] 00020000:00200000:1.0:1789760795.578941:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75657216 chunk: [75657216, 75661312) 75661312, 4096 00020000:00200000:1.0:1789760795.578945:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.578961:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18471, 18471] 00000008:00010000:1.0:1789760795.578974:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.578980:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.578989:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75657216, 75661312) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.578999:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75657216=0x4827000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.579110:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.579113:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75661312 00000020:00010000:1.0:1789760795.579121:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.579125:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.579127:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.579130:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.579132:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.579138:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.579144:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.579158:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760795.579255:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 75657216<>75661312 00000100:00100000:0.0:1789760795.579326:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565595520:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.589461:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985779 gen 3 00000008:00010000:0.0:1789760795.589498:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760795.589504:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.589513:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.589567:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760795.589591:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565595520:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.589783:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18471, 18471] 00000080:00200000:1.0:1789760795.589797:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.589802:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.589813:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75661312 i_blocks 147800 00000020:00200000:1.0:1789760795.589817:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.590033:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.590058:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75661312, count: 4096 00000080:00200000:1.0:1789760795.590064:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75661312, bytes: 4096 00000080:00200000:1.0:1789760795.590071:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.590079:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.590081:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75661312, 75665408) 0 0 00000080:00200000:1.0:1789760795.590085:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.590098:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18472, 18472] 00020000:00200000:1.0:1789760795.590105:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75661312 chunk: [75661312, 75665408) 75665408, 4096 00020000:00200000:1.0:1789760795.590109:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.590123:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18472, 18472] 00000008:00010000:1.0:1789760795.590136:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.590141:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.590262:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75661312, 75665408) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.590274:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75661312=0x4828000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.590321:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.590323:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75665408 00000020:00010000:1.0:1789760795.590330:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.590333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.590336:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.590340:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.590343:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.590349:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.590354:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.590369:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760795.590472:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 75661312<>75665408 00000100:00100000:0.0:1789760795.590536:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565595648:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.602772:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985780 gen 3 00000008:00010000:0.0:1789760795.602803:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760795.602808:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.602865:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.602879:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760795.602897:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565595648:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.603053:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18472, 18472] 00000080:00200000:1.0:1789760795.603067:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.603072:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.603084:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75665408 i_blocks 147808 00000020:00200000:1.0:1789760795.603088:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.603095:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.603169:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75665408, count: 4096 00000080:00200000:1.0:1789760795.603175:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75665408, bytes: 4096 00000080:00200000:1.0:1789760795.603182:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.603187:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.603189:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75665408, 75669504) 0 0 00000080:00200000:1.0:1789760795.603192:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.603201:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18473, 18473] 00020000:00200000:1.0:1789760795.603207:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75665408 chunk: [75665408, 75669504) 75669504, 4096 00020000:00200000:1.0:1789760795.603209:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.603221:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18473, 18473] 00000008:00010000:1.0:1789760795.603231:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.603235:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.603241:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75665408, 75669504) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.603249:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75665408=0x4829000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.603281:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.603283:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75669504 00000020:00010000:1.0:1789760795.603288:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.603290:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.603292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.603294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.603296:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.603300:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.603304:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.603314:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760795.603413:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 75665408<>75669504 00000100:00100000:0.0:1789760795.603548:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760795.603560:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565595776:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.612297:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985781 gen 3 00000008:00010000:0.0:1789760795.612458:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760795.612464:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.612473:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.612600:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760795.612624:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565595776:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.612938:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18473, 18473] 00000080:00200000:1.0:1789760795.612952:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.612957:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.612988:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75669504 i_blocks 147816 00000020:00200000:1.0:1789760795.612991:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.613000:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.613028:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75669504, count: 4096 00000080:00200000:1.0:1789760795.613035:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75669504, bytes: 4096 00000080:00200000:1.0:1789760795.613045:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.613053:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.613055:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75669504, 75673600) 0 0 00000080:00200000:1.0:1789760795.613060:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.613074:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18474, 18474] 00020000:00200000:1.0:1789760795.613082:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75669504 chunk: [75669504, 75673600) 75673600, 4096 00020000:00200000:1.0:1789760795.613087:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.613104:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18474, 18474] 00000008:00010000:1.0:1789760795.613246:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.613251:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.613261:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75669504, 75673600) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.613274:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75669504=0x482a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.613453:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.613457:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75673600 00000020:00010000:1.0:1789760795.613465:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.613469:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.613472:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.613475:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.613478:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.613557:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.613564:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.613578:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760795.613693:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 75669504<>75673600 00000100:00100000:0.0:1789760795.613767:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565595904:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.625590:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985782 gen 3 00000008:00010000:0.0:1789760795.625627:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760795.625633:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.625641:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.625791:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760795.625816:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565595904:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.626413:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18474, 18474] 00000080:00200000:1.0:1789760795.626427:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.626432:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.626444:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75673600 i_blocks 147824 00000020:00200000:1.0:1789760795.626448:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.626455:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.626484:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75673600, count: 4096 00000080:00200000:1.0:1789760795.626491:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75673600, bytes: 4096 00000080:00200000:1.0:1789760795.626498:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.626505:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.626507:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75673600, 75677696) 0 0 00000080:00200000:1.0:1789760795.626511:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.626524:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18475, 18475] 00020000:00200000:1.0:1789760795.626531:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75673600 chunk: [75673600, 75677696) 75677696, 4096 00020000:00200000:1.0:1789760795.626535:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.626550:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18475, 18475] 00000008:00010000:1.0:1789760795.626563:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.626568:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.626577:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75673600, 75677696) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.626587:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75673600=0x482b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.626635:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.626638:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75677696 00000020:00010000:1.0:1789760795.626669:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.626673:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.626676:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.626680:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.626682:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.626688:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.626693:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.626708:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760795.626874:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 75673600<>75677696 00000100:00100000:0.0:1789760795.626946:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565596032:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.641080:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985783 gen 3 00000008:00010000:0.0:1789760795.641124:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760795.641130:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.641139:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.641156:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760795.641181:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565596032:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.641703:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18475, 18475] 00000080:00200000:1.0:1789760795.641717:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.641723:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.641734:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75677696 i_blocks 147832 00000020:00200000:1.0:1789760795.641737:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.641744:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.641768:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75677696, count: 4096 00000080:00200000:1.0:1789760795.641774:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75677696, bytes: 4096 00000080:00200000:1.0:1789760795.641783:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.641790:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.641792:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75677696, 75681792) 0 0 00000080:00200000:1.0:1789760795.641796:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.641809:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18476, 18476] 00020000:00200000:1.0:1789760795.641815:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75677696 chunk: [75677696, 75681792) 75681792, 4096 00020000:00200000:1.0:1789760795.641819:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.641835:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18476, 18476] 00000008:00010000:1.0:1789760795.641848:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.641852:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.641860:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75677696, 75681792) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.641870:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75677696=0x482c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.641917:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.641920:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75681792 00000020:00010000:1.0:1789760795.641928:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.641931:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.641934:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.641937:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.641939:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.641945:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.641951:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.641966:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760795.642164:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 75677696<>75681792 00000100:00100000:0.0:1789760795.642375:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565596160:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.660045:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985784 gen 3 00000008:00010000:0.0:1789760795.660080:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760795.660087:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.660096:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.660113:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760795.660137:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565596160:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.660237:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18476, 18476] 00000080:00200000:1.0:1789760795.660251:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.660255:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.660266:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75681792 i_blocks 147840 00000020:00200000:1.0:1789760795.660270:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.660278:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.660304:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75681792, count: 4096 00000080:00200000:1.0:1789760795.660311:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75681792, bytes: 4096 00000080:00200000:1.0:1789760795.660318:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.660326:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.660328:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75681792, 75685888) 0 0 00000080:00200000:1.0:1789760795.660331:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.660344:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18477, 18477] 00020000:00200000:1.0:1789760795.660351:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75681792 chunk: [75681792, 75685888) 75685888, 4096 00020000:00200000:1.0:1789760795.660355:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.660370:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18477, 18477] 00000008:00010000:1.0:1789760795.660398:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.660402:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.660412:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75681792, 75685888) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.660423:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75681792=0x482d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.660468:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.660471:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75685888 00000020:00010000:1.0:1789760795.660479:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.660483:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.660485:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.660488:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.660490:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.660496:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.660502:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.660515:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760795.660610:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 75681792<>75685888 00000100:00100000:0.0:1789760795.660676:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565596288:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.677977:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985785 gen 3 00000008:00010000:1.0:1789760795.678111:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:1.0:1789760795.678118:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760795.678128:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760795.678205:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:1.0:1789760795.678231:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565596288:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760795.678301:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18477, 18477] 00000080:00200000:2.0:1789760795.678314:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760795.678951:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760795.678978:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75685888 i_blocks 147848 00000020:00200000:2.0:1789760795.678983:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760795.678992:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760795.679015:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75685888, count: 4096 00000080:00200000:2.0:1789760795.679021:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75685888, bytes: 4096 00000080:00200000:2.0:1789760795.679031:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760795.679038:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760795.679041:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75685888, 75689984) 0 0 00000080:00200000:2.0:1789760795.679044:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760795.679058:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18478, 18478] 00020000:00200000:2.0:1789760795.679065:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75685888 chunk: [75685888, 75689984) 75689984, 4096 00020000:00200000:2.0:1789760795.679069:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760795.679084:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18478, 18478] 00000008:00010000:2.0:1789760795.679097:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760795.679102:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760795.679111:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75685888, 75689984) append 0 append lockpos 0 00000080:00200000:2.0:1789760795.679121:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75685888=0x482e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760795.679165:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760795.679167:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75689984 00000020:00010000:2.0:1789760795.679174:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760795.679178:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.679180:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.679583:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760795.679586:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760795.679594:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760795.679601:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760795.679618:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760795.679927:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 75685888<>75689984 00000100:00100000:2.0:1789760795.679972:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760795.679992:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565596416:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760795.695164:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985786 gen 3 00000008:00010000:2.0:1789760795.695200:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760795.695205:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760795.695213:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760795.695275:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760795.695299:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565596416:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.695784:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18478, 18478] 00000080:00200000:1.0:1789760795.695797:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.695803:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.695814:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75689984 i_blocks 147856 00000020:00200000:1.0:1789760795.695818:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.695826:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.695851:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75689984, count: 4096 00000080:00200000:1.0:1789760795.695857:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75689984, bytes: 4096 00000080:00200000:1.0:1789760795.695866:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.695874:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.695876:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75689984, 75694080) 0 0 00000080:00200000:1.0:1789760795.695879:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.695892:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18479, 18479] 00020000:00200000:1.0:1789760795.695899:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75689984 chunk: [75689984, 75694080) 75694080, 4096 00020000:00200000:1.0:1789760795.695903:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.695918:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18479, 18479] 00000008:00010000:1.0:1789760795.695932:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.695936:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.695946:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75689984, 75694080) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.695955:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75689984=0x482f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.695996:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.695998:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75694080 00000020:00010000:1.0:1789760795.696006:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.696009:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.696012:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.696015:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.696017:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.696151:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.696158:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.696173:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760795.696269:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 75689984<>75694080 00000100:00100000:0.0:1789760795.696417:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565596544:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.711098:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985787 gen 3 00000008:00010000:0.0:1789760795.711130:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760795.711136:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.711145:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.711161:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760795.711184:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565596544:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.711332:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18479, 18479] 00000080:00200000:1.0:1789760795.711344:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.711349:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.711360:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75694080 i_blocks 147864 00000020:00200000:1.0:1789760795.711363:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.711819:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.711846:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75694080, count: 4096 00000080:00200000:1.0:1789760795.711853:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75694080, bytes: 4096 00000080:00200000:1.0:1789760795.711862:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.711869:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.711871:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75694080, 75698176) 0 0 00000080:00200000:1.0:1789760795.711875:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.711887:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18480, 18480] 00020000:00200000:1.0:1789760795.711894:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75694080 chunk: [75694080, 75698176) 75698176, 4096 00020000:00200000:1.0:1789760795.711898:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.711912:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18480, 18480] 00000008:00010000:1.0:1789760795.711925:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.711930:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.711939:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75694080, 75698176) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.711950:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75694080=0x4830000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.711994:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.711996:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75698176 00000020:00010000:1.0:1789760795.712003:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.712006:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.712010:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.712013:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.712015:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.712021:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.712026:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.712040:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760795.712144:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 75694080<>75698176 00000100:00100000:0.0:1789760795.713010:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565596672:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.728156:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985788 gen 3 00000008:00010000:0.0:1789760795.728194:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760795.728200:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.728209:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.728227:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760795.728250:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565596672:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.728337:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18480, 18480] 00000080:00200000:1.0:1789760795.728351:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.728356:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.728368:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75698176 i_blocks 147872 00000020:00200000:1.0:1789760795.728372:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.728388:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.728413:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75698176, count: 4096 00000080:00200000:1.0:1789760795.728419:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75698176, bytes: 4096 00000080:00200000:1.0:1789760795.728426:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.728433:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.728435:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75698176, 75702272) 0 0 00000080:00200000:1.0:1789760795.728439:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.728452:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18481, 18481] 00020000:00200000:1.0:1789760795.728460:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75698176 chunk: [75698176, 75702272) 75702272, 4096 00020000:00200000:1.0:1789760795.728463:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.728477:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18481, 18481] 00000008:00010000:1.0:1789760795.728490:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.728495:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.728503:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75698176, 75702272) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.728514:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75698176=0x4831000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.728557:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.728560:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75702272 00000020:00010000:1.0:1789760795.728567:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.728570:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.728573:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.728576:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.728579:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.728585:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.728590:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.728603:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760795.729982:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75698176<>75702272 00000100:00100000:1.0:1789760795.730064:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565596800:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.742103:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985789 gen 3 00000008:00010000:0.0:1789760795.742140:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760795.742146:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.742154:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.742220:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760795.742244:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565596800:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.743833:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18481, 18481] 00000080:00200000:1.0:1789760795.743851:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.743856:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.743870:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75702272 i_blocks 147880 00000020:00200000:1.0:1789760795.743874:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.743884:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.743908:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75702272, count: 4096 00000080:00200000:1.0:1789760795.743914:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75702272, bytes: 4096 00000080:00200000:1.0:1789760795.743923:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.743930:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.743932:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75702272, 75706368) 0 0 00000080:00200000:1.0:1789760795.743936:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.743950:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18482, 18482] 00020000:00200000:1.0:1789760795.743956:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75702272 chunk: [75702272, 75706368) 75706368, 4096 00020000:00200000:1.0:1789760795.743960:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.743975:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18482, 18482] 00000008:00010000:1.0:1789760795.743987:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.743992:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.744000:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75702272, 75706368) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.744011:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75702272=0x4832000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.744057:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.744060:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75706368 00000020:00010000:1.0:1789760795.744067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.744070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.744072:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.744084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.744087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.744094:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.744099:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.744120:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760795.744216:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 75702272<>75706368 00000100:00100000:0.0:1789760795.744288:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565596928:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.757397:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985790 gen 3 00000008:00010000:0.0:1789760795.757431:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760795.757437:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.757445:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.757464:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760795.757486:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565596928:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.757503:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18482, 18482] 00000080:00200000:1.0:1789760795.757518:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.757524:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.757535:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75706368 i_blocks 147888 00000020:00200000:1.0:1789760795.757539:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.757547:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.757574:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75706368, count: 4096 00000080:00200000:1.0:1789760795.757580:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75706368, bytes: 4096 00000080:00200000:1.0:1789760795.757587:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.757594:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.757596:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75706368, 75710464) 0 0 00000080:00200000:1.0:1789760795.757600:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.757612:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18483, 18483] 00020000:00200000:1.0:1789760795.757629:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75706368 chunk: [75706368, 75710464) 75710464, 4096 00020000:00200000:1.0:1789760795.757634:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.757822:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18483, 18483] 00000008:00010000:1.0:1789760795.757837:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.757842:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.757851:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75706368, 75710464) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.757861:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75706368=0x4833000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.757904:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.757907:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75710464 00000020:00010000:1.0:1789760795.757914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.757917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.757920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.757923:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.757925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.757931:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.757936:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.757949:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760795.758048:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 75706368<>75710464 00000100:00100000:0.0:1789760795.758112:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565597056:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.773622:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985791 gen 3 00000008:00010000:0.0:1789760795.773692:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760795.773699:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.773708:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.773718:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760795.773957:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565597056:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.774712:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18483, 18483] 00000080:00200000:1.0:1789760795.774727:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.774732:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.774880:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75710464 i_blocks 147896 00000020:00200000:1.0:1789760795.774885:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.774960:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.774985:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75710464, count: 4096 00000080:00200000:1.0:1789760795.774992:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75710464, bytes: 4096 00000080:00200000:1.0:1789760795.775002:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.775009:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.775011:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75710464, 75714560) 0 0 00000080:00200000:1.0:1789760795.775015:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.775028:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18484, 18484] 00020000:00200000:1.0:1789760795.775035:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75710464 chunk: [75710464, 75714560) 75714560, 4096 00020000:00200000:1.0:1789760795.775039:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.775057:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18484, 18484] 00000008:00010000:1.0:1789760795.775070:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.775075:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.775084:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75710464, 75714560) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.775096:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75710464=0x4834000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.775143:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.775434:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75714560 00000020:00010000:1.0:1789760795.775443:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.777313:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.779301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.779307:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.779310:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.779334:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.779343:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.779770:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760795.780066:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 75710464<>75714560 00000100:00100000:0.0:1789760795.780710:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565597184:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.793037:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985792 gen 3 00000008:00010000:0.0:1789760795.793079:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760795.793086:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.793093:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.793111:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760795.793133:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565597184:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.793256:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18484, 18484] 00000080:00200000:1.0:1789760795.793270:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.793275:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.793287:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75714560 i_blocks 147904 00000020:00200000:1.0:1789760795.793291:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.793299:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.793330:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75714560, count: 4096 00000080:00200000:1.0:1789760795.793337:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75714560, bytes: 4096 00000080:00200000:1.0:1789760795.793345:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.793352:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.793354:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75714560, 75718656) 0 0 00000080:00200000:1.0:1789760795.793358:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.793371:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18485, 18485] 00020000:00200000:1.0:1789760795.793389:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75714560 chunk: [75714560, 75718656) 75718656, 4096 00020000:00200000:1.0:1789760795.793393:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.793408:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18485, 18485] 00000008:00010000:1.0:1789760795.793421:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.793425:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.793434:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75714560, 75718656) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.793445:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75714560=0x4835000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.793553:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.793556:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75718656 00000020:00010000:1.0:1789760795.793563:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.793566:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.793569:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.793572:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.793575:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.793582:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.793588:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.793602:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760795.793731:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 75714560<>75718656 00000100:00100000:0.0:1789760795.793812:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760795.793827:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565597312:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760795.805783:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985793 gen 3 00000008:00010000:0.0:1789760795.805820:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760795.805825:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760795.805833:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760795.805845:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760795.805867:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565597312:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760795.806088:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18485, 18485] 00000080:00200000:2.0:1789760795.806100:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760795.806104:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760795.806114:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75718656 i_blocks 147912 00000020:00200000:2.0:1789760795.806118:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760795.806125:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760795.806151:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75718656, count: 4096 00000080:00200000:2.0:1789760795.806157:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75718656, bytes: 4096 00000080:00200000:2.0:1789760795.806165:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760795.806172:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760795.806174:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75718656, 75722752) 0 0 00000080:00200000:2.0:1789760795.806177:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760795.806191:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18486, 18486] 00020000:00200000:2.0:1789760795.806199:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75718656 chunk: [75718656, 75722752) 75722752, 4096 00020000:00200000:2.0:1789760795.806203:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760795.806219:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18486, 18486] 00000008:00010000:2.0:1789760795.806234:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760795.806238:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760795.806247:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75718656, 75722752) append 0 append lockpos 0 00000080:00200000:2.0:1789760795.806256:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75718656=0x4836000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760795.806469:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760795.806472:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75722752 00000020:00010000:2.0:1789760795.806630:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760795.806635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.806639:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.806642:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760795.806678:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760795.806686:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760795.806691:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760795.806706:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760795.806940:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 75718656<>75722752 00000100:00100000:2.0:1789760795.806985:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565597440:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760795.823044:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985794 gen 3 00000008:00010000:2.0:1789760795.823079:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760795.823085:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760795.823093:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760795.823150:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760795.823175:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565597440:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760795.827043:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18486, 18486] 00000080:00200000:3.0:1789760795.827066:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760795.827071:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760795.827085:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75722752 i_blocks 147920 00000020:00200000:3.0:1789760795.827089:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760795.827104:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760795.827131:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75722752, count: 4096 00000080:00200000:3.0:1789760795.827139:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75722752, bytes: 4096 00000080:00200000:3.0:1789760795.827149:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760795.827158:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760795.827160:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75722752, 75726848) 0 0 00000080:00200000:3.0:1789760795.827164:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760795.827178:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18487, 18487] 00020000:00200000:3.0:1789760795.827185:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75722752 chunk: [75722752, 75726848) 75726848, 4096 00020000:00200000:3.0:1789760795.827189:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760795.827208:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18487, 18487] 00000008:00010000:3.0:1789760795.827222:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760795.827227:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760795.827236:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75722752, 75726848) append 0 append lockpos 0 00000080:00200000:3.0:1789760795.827248:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75722752=0x4837000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760795.827301:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760795.827304:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75726848 00000020:00010000:3.0:1789760795.827311:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:3.0:1789760795.827314:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760795.827317:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760795.827321:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760795.827323:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760795.827330:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760795.827336:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760795.827359:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760795.827875:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 75722752<>75726848 00000100:00100000:3.0:1789760795.827926:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565597568:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760795.843993:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985795 gen 3 00000008:00010000:3.0:1789760795.844029:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:3.0:1789760795.844036:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760795.844045:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760795.844120:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:3.0:1789760795.844145:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565597568:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760795.846723:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18487, 18487] 00000080:00200000:2.0:1789760795.846793:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760795.846799:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760795.846812:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75726848 i_blocks 147928 00000020:00200000:2.0:1789760795.846816:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760795.846829:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760795.846854:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75726848, count: 4096 00000080:00200000:2.0:1789760795.846920:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75726848, bytes: 4096 00000080:00200000:2.0:1789760795.846932:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760795.846940:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760795.846942:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75726848, 75730944) 0 0 00000080:00200000:2.0:1789760795.846946:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760795.846960:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18488, 18488] 00020000:00200000:2.0:1789760795.846967:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75726848 chunk: [75726848, 75730944) 75730944, 4096 00020000:00200000:2.0:1789760795.846971:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760795.846986:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18488, 18488] 00000008:00010000:2.0:1789760795.846999:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760795.847004:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760795.847015:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75726848, 75730944) append 0 append lockpos 0 00000080:00200000:2.0:1789760795.847026:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75726848=0x4838000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760795.847174:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760795.847178:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75730944 00000020:00010000:2.0:1789760795.847186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760795.847189:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.847192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.847195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760795.847198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760795.847204:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760795.847209:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760795.847230:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760795.847413:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 75726848<>75730944 00000100:00100000:3.0:1789760795.847500:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565597696:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760795.860524:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985796 gen 3 00000008:00010000:3.0:1789760795.860563:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760795.860569:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760795.860576:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760795.860597:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760795.860622:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565597696:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760795.860631:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18488, 18488] 00000080:00200000:2.0:1789760795.860676:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760795.860681:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760795.860691:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75730944 i_blocks 147936 00000020:00200000:2.0:1789760795.860695:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760795.860703:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760795.860727:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75730944, count: 4096 00000080:00200000:2.0:1789760795.860733:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75730944, bytes: 4096 00000080:00200000:2.0:1789760795.860740:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760795.860747:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760795.860749:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75730944, 75735040) 0 0 00000080:00200000:2.0:1789760795.860754:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760795.860767:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18489, 18489] 00020000:00200000:2.0:1789760795.860774:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75730944 chunk: [75730944, 75735040) 75735040, 4096 00020000:00200000:2.0:1789760795.860778:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760795.860792:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18489, 18489] 00000008:00010000:2.0:1789760795.860806:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760795.860811:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760795.860819:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75730944, 75735040) append 0 append lockpos 0 00000080:00200000:2.0:1789760795.860830:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75730944=0x4839000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760795.863685:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760795.863693:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75735040 00000020:00010000:2.0:1789760795.863705:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760795.863710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.863713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.863717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760795.863719:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760795.863729:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760795.863737:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760795.863761:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760795.863875:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1f80 - object 0x240000400:5 offset 75730944<>75735040 00000100:00100000:3.0:1789760795.864034:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565597824:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760795.882002:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985797 gen 3 00000008:00010000:3.0:1789760795.882051:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760795.882058:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760795.882067:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760795.882088:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760795.882115:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565597824:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760795.882261:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18489, 18489] 00000080:00200000:2.0:1789760795.882338:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760795.882343:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760795.882356:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75735040 i_blocks 147944 00000020:00200000:2.0:1789760795.882360:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760795.882368:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760795.882401:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75735040, count: 4096 00000080:00200000:2.0:1789760795.882409:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75735040, bytes: 4096 00000080:00200000:2.0:1789760795.882631:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760795.882640:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760795.882685:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75735040, 75739136) 0 0 00000080:00200000:2.0:1789760795.882690:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760795.882704:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18490, 18490] 00020000:00200000:2.0:1789760795.882711:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75735040 chunk: [75735040, 75739136) 75739136, 4096 00020000:00200000:2.0:1789760795.882716:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760795.882734:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18490, 18490] 00000008:00010000:2.0:1789760795.882747:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760795.882751:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760795.882760:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75735040, 75739136) append 0 append lockpos 0 00000080:00200000:2.0:1789760795.882772:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75735040=0x483a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760795.882817:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760795.882820:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75739136 00000020:00010000:2.0:1789760795.882826:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760795.882830:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.882833:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.882836:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760795.882838:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760795.882846:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760795.882851:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760795.883528:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760795.883630:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3b80 - object 0x240000400:5 offset 75735040<>75739136 00000100:00100000:3.0:1789760795.883719:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565597952:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760795.898337:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985798 gen 3 00000008:00010000:3.0:1789760795.898376:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760795.898393:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760795.898402:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760795.898425:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760795.898450:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565597952:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760795.898511:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18490, 18490] 00000080:00200000:1.0:1789760795.898524:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760795.898530:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760795.898541:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75739136 i_blocks 147952 00000020:00200000:1.0:1789760795.898545:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760795.898553:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760795.898581:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75739136, count: 4096 00000080:00200000:1.0:1789760795.898587:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75739136, bytes: 4096 00000080:00200000:1.0:1789760795.898600:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760795.898610:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760795.898614:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75739136, 75743232) 0 0 00000080:00200000:1.0:1789760795.898618:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760795.898633:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18491, 18491] 00020000:00200000:1.0:1789760795.898641:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75739136 chunk: [75739136, 75743232) 75743232, 4096 00020000:00200000:1.0:1789760795.898681:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760795.898698:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18491, 18491] 00000008:00010000:1.0:1789760795.898712:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760795.898717:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760795.898726:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75739136, 75743232) append 0 append lockpos 0 00000080:00200000:1.0:1789760795.898737:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75739136=0x483b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760795.898782:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760795.898785:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75743232 00000020:00010000:1.0:1789760795.898791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760795.898794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.898797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760795.898801:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760795.898803:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760795.898810:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760795.898816:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760795.898830:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760795.899897:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 75739136<>75743232 00000100:00100000:0.0:1789760795.900094:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760795.900123:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565598080:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760795.917097:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985799 gen 3 00000008:00010000:1.0:1789760795.917135:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760795.917142:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760795.917151:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760795.917173:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760795.917200:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565598080:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760795.917226:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18491, 18491] 00000080:00200000:3.0:1789760795.917237:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760795.917242:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760795.917253:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75743232 i_blocks 147960 00000020:00200000:3.0:1789760795.917256:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760795.917264:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760795.917290:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75743232, count: 4096 00000080:00200000:3.0:1789760795.917296:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75743232, bytes: 4096 00000080:00200000:3.0:1789760795.917303:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760795.917311:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760795.917473:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75743232, 75747328) 0 0 00000080:00200000:3.0:1789760795.917479:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760795.917549:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18492, 18492] 00020000:00200000:3.0:1789760795.917556:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75743232 chunk: [75743232, 75747328) 75747328, 4096 00020000:00200000:3.0:1789760795.917561:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760795.917578:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18492, 18492] 00000008:00010000:3.0:1789760795.917594:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760795.917599:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760795.917608:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75743232, 75747328) append 0 append lockpos 0 00000080:00200000:3.0:1789760795.917619:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75743232=0x483c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760795.917872:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760795.917875:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75747328 00000020:00010000:3.0:1789760795.917883:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:3.0:1789760795.917886:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760795.917889:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760795.917891:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760795.917895:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760795.917901:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760795.917906:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760795.917921:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760795.918017:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65880 - object 0x240000400:5 offset 75743232<>75747328 00000100:00100000:2.0:1789760795.918085:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565598208:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760795.934899:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985800 gen 3 00000008:00010000:2.0:1789760795.934939:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760795.934945:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760795.934954:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760795.934970:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760795.934993:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565598208:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760795.935143:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18492, 18492] 00000080:00200000:3.0:1789760795.935303:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760795.935308:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760795.935320:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75747328 i_blocks 147968 00000020:00200000:3.0:1789760795.935323:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760795.935331:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760795.935357:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75747328, count: 4096 00000080:00200000:3.0:1789760795.935364:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75747328, bytes: 4096 00000080:00200000:3.0:1789760795.935373:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760795.935389:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760795.935392:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75747328, 75751424) 0 0 00000080:00200000:3.0:1789760795.935395:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760795.942327:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18493, 18493] 00020000:00200000:2.0:1789760795.942342:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75747328 chunk: [75747328, 75751424) 75751424, 4096 00020000:00200000:2.0:1789760795.942346:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760795.942371:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18493, 18493] 00000008:00010000:2.0:1789760795.942400:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760795.942406:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760795.942418:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75747328, 75751424) append 0 append lockpos 0 00000080:00200000:2.0:1789760795.942430:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75747328=0x483d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760795.942481:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760795.942709:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75751424 00000020:00010000:2.0:1789760795.942717:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760795.942720:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.942790:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.942794:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760795.942797:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760795.942805:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760795.942810:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760795.942857:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760795.942960:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0000 - object 0x240000400:5 offset 75747328<>75751424 00000100:00100000:3.0:1789760795.943037:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565598336:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760795.964066:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985801 gen 3 00000008:00010000:3.0:1789760795.964105:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760795.964112:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760795.964120:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760795.964141:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760795.964181:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565598336:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760795.965083:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18493, 18493] 00000080:00200000:2.0:1789760795.965098:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760795.965103:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760795.965114:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75751424 i_blocks 147976 00000020:00200000:2.0:1789760795.965118:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760795.965127:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760795.965170:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75751424, count: 4096 00000080:00200000:2.0:1789760795.965176:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75751424, bytes: 4096 00000080:00200000:2.0:1789760795.965185:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760795.965194:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760795.965196:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75751424, 75755520) 0 0 00000080:00200000:2.0:1789760795.965200:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760795.965213:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18494, 18494] 00020000:00200000:2.0:1789760795.965220:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75751424 chunk: [75751424, 75755520) 75755520, 4096 00020000:00200000:2.0:1789760795.965224:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760795.965240:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18494, 18494] 00000008:00010000:2.0:1789760795.965254:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760795.965259:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760795.965269:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75751424, 75755520) append 0 append lockpos 0 00000080:00200000:2.0:1789760795.965279:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75751424=0x483e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760795.965693:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760795.965696:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75755520 00000020:00010000:2.0:1789760795.965703:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760795.965707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.965709:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.965712:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760795.965714:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760795.965721:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760795.965736:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760795.965752:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760795.965849:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2300 - object 0x240000400:5 offset 75751424<>75755520 00000100:00100000:3.0:1789760795.966199:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565598464:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760795.980847:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985802 gen 3 00000008:00010000:3.0:1789760795.980884:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:3.0:1789760795.980891:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760795.980900:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760795.980919:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:3.0:1789760795.980944:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565598464:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760795.981025:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18494, 18494] 00000080:00200000:2.0:1789760795.981039:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760795.981044:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760795.981054:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75755520 i_blocks 147984 00000020:00200000:2.0:1789760795.981058:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760795.981066:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760795.981094:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75755520, count: 4096 00000080:00200000:2.0:1789760795.981101:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75755520, bytes: 4096 00000080:00200000:2.0:1789760795.981108:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760795.981116:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760795.981118:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75755520, 75759616) 0 0 00000080:00200000:2.0:1789760795.981122:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760795.981136:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18495, 18495] 00020000:00200000:2.0:1789760795.981143:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75755520 chunk: [75755520, 75759616) 75759616, 4096 00020000:00200000:2.0:1789760795.981148:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760795.981164:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18495, 18495] 00000008:00010000:2.0:1789760795.981180:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760795.981186:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760795.981195:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75755520, 75759616) append 0 append lockpos 0 00000080:00200000:2.0:1789760795.981206:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75755520=0x483f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760795.981250:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760795.981253:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75759616 00000020:00010000:2.0:1789760795.981261:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760795.981264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.981268:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760795.981272:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760795.981274:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760795.981281:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760795.981286:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760795.981301:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760795.981412:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 75755520<>75759616 00000100:00100000:3.0:1789760795.981538:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565598592:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.006158:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985803 gen 3 00000008:00010000:2.0:1789760796.008852:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760796.008863:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760796.008871:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760796.008977:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760796.009002:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565598592:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.011068:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18495, 18495] 00000080:00200000:0.0:1789760796.011089:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.011094:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.011107:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75759616 i_blocks 147992 00000020:00200000:0.0:1789760796.011111:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.011123:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.011148:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75759616, count: 4096 00000080:00200000:0.0:1789760796.011154:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75759616, bytes: 4096 00000080:00200000:0.0:1789760796.011166:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.011173:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.011175:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75759616, 75763712) 0 0 00000080:00200000:0.0:1789760796.011179:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.011194:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18496, 18496] 00020000:00200000:0.0:1789760796.011202:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75759616 chunk: [75759616, 75763712) 75763712, 4096 00020000:00200000:0.0:1789760796.011206:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.011222:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18496, 18496] 00000008:00010000:0.0:1789760796.011237:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.011242:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.011251:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75759616, 75763712) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.011264:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75759616=0x4840000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.011313:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.011316:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75763712 00000020:00010000:0.0:1789760796.011322:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.011325:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.011328:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.011332:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.011334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.011340:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.011346:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.011365:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:0.0:1789760796.012105:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42300 - object 0x240000400:5 offset 75759616<>75763712 00000100:00100000:1.0:1789760796.012180:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565598720:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.031881:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985804 gen 3 00000008:00010000:1.0:1789760796.032074:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760796.032083:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.032092:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.032118:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:1.0:1789760796.032149:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565598720:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.032254:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18496, 18496] 00000080:00200000:0.0:1789760796.032268:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.032273:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.032287:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75763712 i_blocks 148000 00000020:00200000:0.0:1789760796.032290:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.032299:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.032326:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75763712, count: 4096 00000080:00200000:0.0:1789760796.032334:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75763712, bytes: 4096 00000080:00200000:0.0:1789760796.032347:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.032356:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.032358:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75763712, 75767808) 0 0 00000080:00200000:0.0:1789760796.032362:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.032374:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18497, 18497] 00020000:00200000:0.0:1789760796.032393:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75763712 chunk: [75763712, 75767808) 75767808, 4096 00020000:00200000:0.0:1789760796.032397:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.032413:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18497, 18497] 00000008:00010000:0.0:1789760796.032425:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.032429:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.032824:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75763712, 75767808) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.032837:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75763712=0x4841000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.032886:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.032889:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75767808 00000020:00010000:0.0:1789760796.032896:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.032899:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.032902:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.032905:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.032908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.033055:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.033062:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.033078:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760796.033177:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41c00 - object 0x240000400:5 offset 75763712<>75767808 00000100:00100000:1.0:1789760796.033550:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565598848:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.050702:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985805 gen 3 00000008:00010000:1.0:1789760796.050736:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760796.050741:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.050749:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.050757:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760796.050778:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565598848:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.050864:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18497, 18497] 00000080:00200000:0.0:1789760796.050877:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.050882:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.050894:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75767808 i_blocks 148008 00000020:00200000:0.0:1789760796.050897:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.050906:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.050990:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75767808, count: 4096 00000080:00200000:0.0:1789760796.050997:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75767808, bytes: 4096 00000080:00200000:0.0:1789760796.051004:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.051011:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.051014:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75767808, 75771904) 0 0 00000080:00200000:0.0:1789760796.051017:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.051029:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18498, 18498] 00020000:00200000:0.0:1789760796.051036:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75767808 chunk: [75767808, 75771904) 75771904, 4096 00020000:00200000:0.0:1789760796.051039:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.051052:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18498, 18498] 00000008:00010000:0.0:1789760796.051064:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.051069:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.051076:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75767808, 75771904) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.051086:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75767808=0x4842000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.051125:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.051127:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75771904 00000020:00010000:0.0:1789760796.051133:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.051136:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.051139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.051142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.051145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.051150:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.051155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.051166:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874da80 00000008:00100000:0.0:1789760796.051247:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40380 - object 0x240000400:5 offset 75767808<>75771904 00000100:00100000:1.0:1789760796.051332:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760796.051344:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565598976:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.067820:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985806 gen 3 00000008:00010000:1.0:1789760796.067858:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874da80 00000020:00200000:1.0:1789760796.067864:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.067873:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.067890:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874da80 00000100:00100000:1.0:1789760796.067911:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565598976:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.068021:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18498, 18498] 00000080:00200000:0.0:1789760796.068036:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.068041:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.068052:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75771904 i_blocks 148016 00000020:00200000:0.0:1789760796.068056:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.068065:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.068092:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75771904, count: 4096 00000080:00200000:0.0:1789760796.068098:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75771904, bytes: 4096 00000080:00200000:0.0:1789760796.068105:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.068113:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.068114:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75771904, 75776000) 0 0 00000080:00200000:0.0:1789760796.068118:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.068130:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18499, 18499] 00020000:00200000:0.0:1789760796.068137:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75771904 chunk: [75771904, 75776000) 75776000, 4096 00020000:00200000:0.0:1789760796.068140:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.068154:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18499, 18499] 00000008:00010000:0.0:1789760796.068166:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.068170:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.068179:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75771904, 75776000) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.068189:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75771904=0x4843000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.068227:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.068230:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75776000 00000020:00010000:0.0:1789760796.068237:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.068240:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.068243:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.068247:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.068249:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.068256:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.068261:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.068401:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d540 00000008:00100000:1.0:1789760796.068546:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 75771904<>75776000 00000100:00100000:1.0:1789760796.068590:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565599104:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.086246:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985807 gen 3 00000008:00010000:1.0:1789760796.086422:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d540 00000020:00200000:1.0:1789760796.086429:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.086438:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.086513:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d540 00000100:00100000:1.0:1789760796.086539:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565599104:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.089822:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18499, 18499] 00000080:00200000:0.0:1789760796.089841:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.089847:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.089862:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75776000 i_blocks 148024 00000020:00200000:0.0:1789760796.089866:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.089880:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.089906:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75776000, count: 4096 00000080:00200000:0.0:1789760796.089913:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75776000, bytes: 4096 00000080:00200000:0.0:1789760796.089926:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.089934:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.089936:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75776000, 75780096) 0 0 00000080:00200000:0.0:1789760796.089940:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.089952:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18500, 18500] 00020000:00200000:0.0:1789760796.089959:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75776000 chunk: [75776000, 75780096) 75780096, 4096 00020000:00200000:0.0:1789760796.089963:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.090036:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18500, 18500] 00000008:00010000:0.0:1789760796.090051:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.090117:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.090129:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75776000, 75780096) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.090141:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75776000=0x4844000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.090195:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.090198:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75780096 00000020:00010000:0.0:1789760796.090205:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.090209:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.090211:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.090214:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.090217:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.090224:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.090229:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.090311:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dcc0 00000008:00100000:0.0:1789760796.090889:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40e00 - object 0x240000400:5 offset 75776000<>75780096 00000100:00100000:1.0:1789760796.090944:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760796.091495:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565599232:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.112204:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985808 gen 3 00000008:00010000:1.0:1789760796.112242:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dcc0 00000020:00200000:1.0:1789760796.112249:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.112266:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.112289:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dcc0 00000100:00100000:1.0:1789760796.112321:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565599232:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.112505:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18500, 18500] 00000080:00200000:2.0:1789760796.112519:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.112523:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.112535:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75780096 i_blocks 148032 00000020:00200000:2.0:1789760796.112538:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.112547:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.112574:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75780096, count: 4096 00000080:00200000:2.0:1789760796.112580:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75780096, bytes: 4096 00000080:00200000:2.0:1789760796.112588:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.112596:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.112598:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75780096, 75784192) 0 0 00000080:00200000:2.0:1789760796.112603:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.112628:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18501, 18501] 00020000:00200000:2.0:1789760796.112635:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75780096 chunk: [75780096, 75784192) 75784192, 4096 00020000:00200000:2.0:1789760796.112639:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.112690:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18501, 18501] 00000008:00010000:2.0:1789760796.112704:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.112708:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.112717:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75780096, 75784192) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.112727:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75780096=0x4845000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.112769:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.112772:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75784192 00000020:00010000:2.0:1789760796.112778:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.112781:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.112784:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.112787:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.112789:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.112795:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.112801:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.112814:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760796.112903:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 75780096<>75784192 00000100:00100000:2.0:1789760796.112943:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760796.112956:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565599360:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760796.127243:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985809 gen 3 00000008:00010000:2.0:1789760796.127288:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760796.127294:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760796.127304:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760796.127325:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760796.127348:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565599360:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.127449:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18501, 18501] 00000080:00200000:1.0:1789760796.127462:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.127466:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.127478:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75784192 i_blocks 148040 00000020:00200000:1.0:1789760796.127481:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.127489:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.127513:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75784192, count: 4096 00000080:00200000:1.0:1789760796.127519:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75784192, bytes: 4096 00000080:00200000:1.0:1789760796.127527:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.127533:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.127535:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75784192, 75788288) 0 0 00000080:00200000:1.0:1789760796.127538:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.127549:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18502, 18502] 00020000:00200000:1.0:1789760796.127556:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75784192 chunk: [75784192, 75788288) 75788288, 4096 00020000:00200000:1.0:1789760796.127560:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.127573:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18502, 18502] 00000008:00010000:1.0:1789760796.127584:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.127589:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.127597:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75784192, 75788288) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.127607:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75784192=0x4846000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.127680:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.127683:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75788288 00000020:00010000:1.0:1789760796.127689:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.127693:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.127695:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.127697:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.127700:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.127706:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.127711:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.127724:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760796.127822:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 75784192<>75788288 00000100:00100000:0.0:1789760796.128018:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760796.128035:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565599488:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.143020:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985810 gen 3 00000008:00010000:1.0:1789760796.143071:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760796.143083:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.143094:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.143117:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760796.143140:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565599488:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.143694:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18502, 18502] 00000080:00200000:0.0:1789760796.143717:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.143722:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.143736:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75788288 i_blocks 148048 00000020:00200000:0.0:1789760796.143740:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.143749:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.143775:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75788288, count: 4096 00000080:00200000:0.0:1789760796.143782:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75788288, bytes: 4096 00000080:00200000:0.0:1789760796.143791:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.143798:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.143800:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75788288, 75792384) 0 0 00000080:00200000:0.0:1789760796.143804:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.143816:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18503, 18503] 00020000:00200000:0.0:1789760796.143823:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75788288 chunk: [75788288, 75792384) 75792384, 4096 00020000:00200000:0.0:1789760796.143827:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.143841:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18503, 18503] 00000008:00010000:0.0:1789760796.143855:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.143859:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.143868:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75788288, 75792384) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.143878:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75788288=0x4847000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.143929:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.143932:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75792384 00000020:00010000:0.0:1789760796.143938:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.143942:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.143944:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.143948:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.143950:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.143957:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.143963:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.143977:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874db40 00000008:00100000:0.0:1789760796.144088:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43b80 - object 0x240000400:5 offset 75788288<>75792384 00000100:00100000:0.0:1789760796.144118:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760796.144134:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565599616:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.154737:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985811 gen 3 00000008:00010000:1.0:1789760796.154772:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874db40 00000020:00200000:1.0:1789760796.154778:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.154786:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.154803:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874db40 00000100:00100000:1.0:1789760796.154826:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565599616:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.154915:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18503, 18503] 00000080:00200000:0.0:1789760796.154928:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.154933:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.154945:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75792384 i_blocks 148056 00000020:00200000:0.0:1789760796.154948:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.154957:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.154983:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75792384, count: 4096 00000080:00200000:0.0:1789760796.154989:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75792384, bytes: 4096 00000080:00200000:0.0:1789760796.154997:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.155004:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.155006:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75792384, 75796480) 0 0 00000080:00200000:0.0:1789760796.155009:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.155022:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18504, 18504] 00020000:00200000:0.0:1789760796.155029:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75792384 chunk: [75792384, 75796480) 75796480, 4096 00020000:00200000:0.0:1789760796.155032:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.155046:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18504, 18504] 00000008:00010000:0.0:1789760796.155060:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.155064:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.155072:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75792384, 75796480) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.155090:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75792384=0x4848000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.155134:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.155136:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75796480 00000020:00010000:0.0:1789760796.155142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.155145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.155147:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.155150:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.155152:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.155158:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.155163:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.155177:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874de40 00000008:00100000:0.0:1789760796.155272:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d42680 - object 0x240000400:5 offset 75792384<>75796480 00000100:00100000:1.0:1789760796.155739:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565599744:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.166619:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985812 gen 3 00000008:00010000:1.0:1789760796.166681:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874de40 00000020:00200000:1.0:1789760796.166686:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.166694:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.166702:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874de40 00000100:00100000:1.0:1789760796.166722:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565599744:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.166795:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18504, 18504] 00000080:00200000:0.0:1789760796.166807:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.166812:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.166822:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75796480 i_blocks 148064 00000020:00200000:0.0:1789760796.166825:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.166834:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.166859:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75796480, count: 4096 00000080:00200000:0.0:1789760796.166865:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75796480, bytes: 4096 00000080:00200000:0.0:1789760796.166872:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.166879:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.166882:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75796480, 75800576) 0 0 00000080:00200000:0.0:1789760796.166885:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.166897:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18505, 18505] 00020000:00200000:0.0:1789760796.166903:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75796480 chunk: [75796480, 75800576) 75800576, 4096 00020000:00200000:0.0:1789760796.166906:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.166919:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18505, 18505] 00000008:00010000:0.0:1789760796.166931:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.166935:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.166943:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75796480, 75800576) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.166952:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75796480=0x4849000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.166991:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.166994:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75800576 00000020:00010000:0.0:1789760796.167000:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.167003:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.167006:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.167009:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.167011:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.167017:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.167022:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.167035:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874dd80 00000008:00100000:0.0:1789760796.167117:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40000 - object 0x240000400:5 offset 75796480<>75800576 00000100:00100000:1.0:1789760796.167169:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565599872:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.183060:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985813 gen 3 00000008:00010000:1.0:1789760796.183098:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874dd80 00000020:00200000:1.0:1789760796.183103:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.183114:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.183182:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874dd80 00000100:00100000:1.0:1789760796.183207:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565599872:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.183371:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18505, 18505] 00000080:00200000:0.0:1789760796.183395:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.183401:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.183412:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75800576 i_blocks 148072 00000020:00200000:0.0:1789760796.183416:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.183426:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.183457:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75800576, count: 4096 00000080:00200000:0.0:1789760796.183463:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75800576, bytes: 4096 00000080:00200000:0.0:1789760796.183471:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.183478:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.183480:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75800576, 75804672) 0 0 00000080:00200000:0.0:1789760796.183494:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.183508:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18506, 18506] 00020000:00200000:0.0:1789760796.183515:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75800576 chunk: [75800576, 75804672) 75804672, 4096 00020000:00200000:0.0:1789760796.183519:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.183536:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18506, 18506] 00000008:00010000:0.0:1789760796.183550:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.183556:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.183565:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75800576, 75804672) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.183577:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75800576=0x484a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.183625:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.183628:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75804672 00000020:00010000:0.0:1789760796.183635:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.183638:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.183641:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.183940:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.183943:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.183951:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.183957:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.183973:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d780 00000008:00100000:0.0:1789760796.184073:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d40700 - object 0x240000400:5 offset 75800576<>75804672 00000100:00100000:1.0:1789760796.184134:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565600000:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760796.186816:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760796.197887:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985814 gen 3 00000008:00010000:1.0:1789760796.197929:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d780 00000020:00200000:1.0:1789760796.197935:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.197944:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.197999:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d780 00000100:00100000:1.0:1789760796.198022:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565600000:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.199598:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18506, 18506] 00000080:00200000:0.0:1789760796.199615:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.199620:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.199633:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75804672 i_blocks 148080 00000020:00200000:0.0:1789760796.199637:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.199662:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.199688:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75804672, count: 4096 00000080:00200000:0.0:1789760796.199695:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75804672, bytes: 4096 00000080:00200000:0.0:1789760796.199705:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.199712:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.199714:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75804672, 75808768) 0 0 00000080:00200000:0.0:1789760796.199718:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.199732:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18507, 18507] 00020000:00200000:0.0:1789760796.199739:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75804672 chunk: [75804672, 75808768) 75808768, 4096 00020000:00200000:0.0:1789760796.199743:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.199761:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18507, 18507] 00000008:00010000:0.0:1789760796.199775:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.199780:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.199789:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75804672, 75808768) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.199800:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75804672=0x484b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.199853:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.199856:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75808768 00000020:00010000:0.0:1789760796.199864:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.199868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.199870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.199873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.199876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.199882:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.199887:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.199907:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d300 00000008:00100000:0.0:1789760796.201410:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43100 - object 0x240000400:5 offset 75804672<>75808768 00000100:00100000:1.0:1789760796.201612:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565600128:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.215806:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985815 gen 3 00000008:00010000:1.0:1789760796.215840:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d300 00000020:00200000:1.0:1789760796.215845:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.215853:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.215870:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d300 00000100:00100000:1.0:1789760796.215892:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565600128:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.215971:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18507, 18507] 00000080:00200000:0.0:1789760796.215985:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.215990:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.216002:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75808768 i_blocks 148088 00000020:00200000:0.0:1789760796.216005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.216014:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.216037:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75808768, count: 4096 00000080:00200000:0.0:1789760796.216043:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75808768, bytes: 4096 00000080:00200000:0.0:1789760796.216050:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.216056:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.216058:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75808768, 75812864) 0 0 00000080:00200000:0.0:1789760796.216062:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.216074:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18508, 18508] 00020000:00200000:0.0:1789760796.216081:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75808768 chunk: [75808768, 75812864) 75812864, 4096 00020000:00200000:0.0:1789760796.216085:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.216100:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18508, 18508] 00000008:00010000:0.0:1789760796.216115:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.216120:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.216129:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75808768, 75812864) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.216140:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75808768=0x484c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.216182:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.216184:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75812864 00000020:00010000:0.0:1789760796.216191:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.216195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.216197:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.216201:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.216203:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.216209:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.216215:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.216229:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d000 00000008:00100000:0.0:1789760796.216322:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41f80 - object 0x240000400:5 offset 75808768<>75812864 00000100:00100000:1.0:1789760796.216424:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565600256:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.230193:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985816 gen 3 00000008:00010000:1.0:1789760796.230229:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d000 00000020:00200000:1.0:1789760796.230235:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.230244:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.230305:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d000 00000100:00100000:1.0:1789760796.230338:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565600256:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.230892:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18508, 18508] 00000080:00200000:0.0:1789760796.230909:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.230914:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.230927:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75812864 i_blocks 148096 00000020:00200000:0.0:1789760796.230931:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.230939:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.230964:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75812864, count: 4096 00000080:00200000:0.0:1789760796.230971:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75812864, bytes: 4096 00000080:00200000:0.0:1789760796.230979:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.230985:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.230988:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75812864, 75816960) 0 0 00000080:00200000:0.0:1789760796.230991:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.231005:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18509, 18509] 00020000:00200000:0.0:1789760796.231013:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75812864 chunk: [75812864, 75816960) 75816960, 4096 00020000:00200000:0.0:1789760796.231016:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.231102:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18509, 18509] 00000008:00010000:0.0:1789760796.231118:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.231123:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.231132:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75812864, 75816960) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.231142:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75812864=0x484d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.231185:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.231188:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75816960 00000020:00010000:0.0:1789760796.231195:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.231198:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.231200:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.231203:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.231206:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.231212:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.231217:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.231233:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d0c0 00000008:00100000:0.0:1789760796.231704:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 75812864<>75816960 00000100:00100000:1.0:1789760796.231820:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565600384:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.259085:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985817 gen 3 00000008:00010000:1.0:1789760796.259120:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d0c0 00000020:00200000:1.0:1789760796.259126:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.259134:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.259152:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d0c0 00000100:00100000:1.0:1789760796.259174:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565600384:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.259260:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18509, 18509] 00000080:00200000:0.0:1789760796.259273:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.259278:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.259291:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75816960 i_blocks 148104 00000020:00200000:0.0:1789760796.259295:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.259304:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.259329:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75816960, count: 4096 00000080:00200000:0.0:1789760796.259336:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75816960, bytes: 4096 00000080:00200000:0.0:1789760796.259344:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.259351:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.259353:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75816960, 75821056) 0 0 00000080:00200000:0.0:1789760796.259357:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.259370:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18510, 18510] 00020000:00200000:0.0:1789760796.259387:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75816960 chunk: [75816960, 75821056) 75821056, 4096 00020000:00200000:0.0:1789760796.259391:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.259407:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18510, 18510] 00000008:00010000:0.0:1789760796.259420:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.259433:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.259442:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75816960, 75821056) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.259455:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75816960=0x484e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.259497:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.259500:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75821056 00000020:00010000:0.0:1789760796.259507:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.259510:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.259512:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.259515:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.259518:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.259523:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.259529:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.259542:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d840 00000008:00100000:0.0:1789760796.259680:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 75816960<>75821056 00000100:00100000:1.0:1789760796.259762:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565600512:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.283658:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985818 gen 3 00000008:00010000:1.0:1789760796.283696:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d840 00000020:00200000:1.0:1789760796.283702:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.283710:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.283728:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d840 00000100:00100000:1.0:1789760796.283748:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565600512:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.283825:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18510, 18510] 00000080:00200000:0.0:1789760796.283839:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.283845:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.283856:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75821056 i_blocks 148112 00000020:00200000:0.0:1789760796.283860:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.283868:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.283895:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75821056, count: 4096 00000080:00200000:0.0:1789760796.283901:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75821056, bytes: 4096 00000080:00200000:0.0:1789760796.283909:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.283916:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.283918:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75821056, 75825152) 0 0 00000080:00200000:0.0:1789760796.283922:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.283934:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18511, 18511] 00020000:00200000:0.0:1789760796.283941:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75821056 chunk: [75821056, 75825152) 75825152, 4096 00020000:00200000:0.0:1789760796.283945:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.283961:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18511, 18511] 00000008:00010000:0.0:1789760796.283973:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.283978:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.283987:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75821056, 75825152) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.283998:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75821056=0x484f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.284040:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.284043:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75825152 00000020:00010000:0.0:1789760796.284050:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.284053:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.284056:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.284059:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.284062:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.284067:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.284073:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.284086:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874df00 00000008:00100000:0.0:1789760796.284181:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 75821056<>75825152 00000100:00100000:1.0:1789760796.284331:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565600640:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.301753:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985819 gen 3 00000008:00010000:1.0:1789760796.301788:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874df00 00000020:00200000:1.0:1789760796.301794:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.301802:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.301869:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874df00 00000100:00100000:1.0:1789760796.301892:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565600640:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.301983:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18511, 18511] 00000080:00200000:0.0:1789760796.301996:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.302001:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.302012:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75825152 i_blocks 148120 00000020:00200000:0.0:1789760796.302015:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.302025:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.302050:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75825152, count: 4096 00000080:00200000:0.0:1789760796.302057:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75825152, bytes: 4096 00000080:00200000:0.0:1789760796.302064:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.302072:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.302074:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75825152, 75829248) 0 0 00000080:00200000:0.0:1789760796.302078:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.302091:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18512, 18512] 00020000:00200000:0.0:1789760796.302097:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75825152 chunk: [75825152, 75829248) 75829248, 4096 00020000:00200000:0.0:1789760796.302101:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.302115:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18512, 18512] 00000008:00010000:0.0:1789760796.302131:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.302135:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.302145:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75825152, 75829248) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.302156:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75825152=0x4850000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.303176:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.303183:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75829248 00000020:00010000:0.0:1789760796.303192:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.303196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.303199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.303202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.303205:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.303213:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.303219:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.303240:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d480 00000008:00100000:0.0:1789760796.303349:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 75825152<>75829248 00000100:00100000:1.0:1789760796.303582:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565600768:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.319062:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985820 gen 3 00000008:00010000:1.0:1789760796.319102:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d480 00000020:00200000:1.0:1789760796.319109:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.319118:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.319135:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d480 00000100:00100000:1.0:1789760796.319159:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565600768:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.319248:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18512, 18512] 00000080:00200000:0.0:1789760796.319261:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.319266:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.319277:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75829248 i_blocks 148128 00000020:00200000:0.0:1789760796.319281:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.319290:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.319326:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75829248, count: 4096 00000080:00200000:0.0:1789760796.319332:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75829248, bytes: 4096 00000080:00200000:0.0:1789760796.319339:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.319347:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.319349:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75829248, 75833344) 0 0 00000080:00200000:0.0:1789760796.319353:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.319366:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18513, 18513] 00020000:00200000:0.0:1789760796.319373:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75829248 chunk: [75829248, 75833344) 75833344, 4096 00020000:00200000:0.0:1789760796.319386:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.319402:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18513, 18513] 00000008:00010000:0.0:1789760796.319416:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.319421:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.319432:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75829248, 75833344) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.319444:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75829248=0x4851000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.319567:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.319570:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75833344 00000020:00010000:0.0:1789760796.319578:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.319581:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.319584:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.319587:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.319590:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.319596:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.319603:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.319616:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760796.319755:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41180 - object 0x240000400:5 offset 75829248<>75833344 00000100:00100000:1.0:1789760796.319967:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565600896:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.330707:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985821 gen 3 00000008:00010000:1.0:1789760796.330744:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:1.0:1789760796.330750:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.330758:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.330882:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:1.0:1789760796.330907:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565600896:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.331368:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18513, 18513] 00000080:00200000:0.0:1789760796.331390:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.331396:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.331408:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75833344 i_blocks 148136 00000020:00200000:0.0:1789760796.331411:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.331421:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.331448:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75833344, count: 4096 00000080:00200000:0.0:1789760796.331454:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75833344, bytes: 4096 00000080:00200000:0.0:1789760796.331463:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.331471:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.331473:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75833344, 75837440) 0 0 00000080:00200000:0.0:1789760796.331477:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.331555:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18514, 18514] 00020000:00200000:0.0:1789760796.331563:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75833344 chunk: [75833344, 75837440) 75837440, 4096 00020000:00200000:0.0:1789760796.331567:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.331583:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18514, 18514] 00000008:00010000:0.0:1789760796.331598:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.331603:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.331612:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75833344, 75837440) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.331623:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75833344=0x4852000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.331894:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.331898:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75837440 00000020:00010000:0.0:1789760796.331907:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.331911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.331914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.331917:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.331920:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.331927:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.331933:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.331952:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:0.0:1789760796.332058:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43800 - object 0x240000400:5 offset 75833344<>75837440 00000100:00100000:1.0:1789760796.332128:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565601024:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.343422:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985822 gen 3 00000008:00010000:1.0:1789760796.343460:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760796.343467:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.343474:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.343529:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760796.343555:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565601024:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.344189:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18514, 18514] 00000080:00200000:0.0:1789760796.344201:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.344207:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.344218:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75837440 i_blocks 148144 00000020:00200000:0.0:1789760796.344221:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.344232:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.344258:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75837440, count: 4096 00000080:00200000:0.0:1789760796.344264:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75837440, bytes: 4096 00000080:00200000:0.0:1789760796.344273:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.344280:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.344282:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75837440, 75841536) 0 0 00000080:00200000:0.0:1789760796.344286:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.344299:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18515, 18515] 00020000:00200000:0.0:1789760796.344306:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75837440 chunk: [75837440, 75841536) 75841536, 4096 00020000:00200000:0.0:1789760796.344309:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.344392:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18515, 18515] 00000008:00010000:0.0:1789760796.344405:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.344410:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.344448:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75837440, 75841536) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.344458:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75837440=0x4853000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.344549:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.344552:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75841536 00000020:00010000:0.0:1789760796.344559:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.344563:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.344566:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.344569:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.344571:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.344577:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.344582:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.344598:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d600 00000008:00100000:0.0:1789760796.344959:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d43480 - object 0x240000400:5 offset 75837440<>75841536 00000100:00100000:1.0:1789760796.345014:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760796.345084:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565601152:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.355007:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985823 gen 3 00000008:00010000:1.0:1789760796.355045:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d600 00000020:00200000:1.0:1789760796.355052:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.355060:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.355084:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d600 00000100:00100000:1.0:1789760796.355119:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565601152:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.355194:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18515, 18515] 00000080:00200000:2.0:1789760796.355207:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.355212:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.355223:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75841536 i_blocks 148152 00000020:00200000:2.0:1789760796.355227:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.355237:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.355262:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75841536, count: 4096 00000080:00200000:2.0:1789760796.355267:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75841536, bytes: 4096 00000080:00200000:2.0:1789760796.355275:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.355282:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.355284:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75841536, 75845632) 0 0 00000080:00200000:2.0:1789760796.355288:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.355300:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18516, 18516] 00020000:00200000:2.0:1789760796.355307:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75841536 chunk: [75841536, 75845632) 75845632, 4096 00020000:00200000:2.0:1789760796.355311:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.355326:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18516, 18516] 00000008:00010000:2.0:1789760796.355339:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.355343:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.355351:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75841536, 75845632) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.355361:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75841536=0x4854000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.355417:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.355420:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75845632 00000020:00010000:2.0:1789760796.355426:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.355430:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.355432:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.355435:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.355438:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.355444:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.355450:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.355462:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760796.355555:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 75841536<>75845632 00000100:00100000:3.0:1789760796.355681:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760796.355698:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565601280:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.375774:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985824 gen 3 00000008:00010000:2.0:1789760796.375882:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:2.0:1789760796.375889:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760796.375897:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760796.375932:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:2.0:1789760796.375957:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565601280:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.376060:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18516, 18516] 00000080:00200000:1.0:1789760796.376071:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.376076:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.376095:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75845632 i_blocks 148160 00000020:00200000:1.0:1789760796.376098:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.376112:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.376143:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75845632, count: 4096 00000080:00200000:1.0:1789760796.376149:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75845632, bytes: 4096 00000080:00200000:1.0:1789760796.376160:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.376167:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.376169:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75845632, 75849728) 0 0 00000080:00200000:1.0:1789760796.376172:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.376184:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18517, 18517] 00020000:00200000:1.0:1789760796.376192:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75845632 chunk: [75845632, 75849728) 75849728, 4096 00020000:00200000:1.0:1789760796.376195:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.376208:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18517, 18517] 00000008:00010000:1.0:1789760796.376220:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.376225:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.376233:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75845632, 75849728) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.376244:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75845632=0x4855000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.376283:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.376285:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75849728 00000020:00010000:1.0:1789760796.376292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.376295:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.376298:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.376301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.376303:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.376308:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.376313:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.376325:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760796.376426:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 75845632<>75849728 00000100:00100000:1.0:1789760796.376467:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760796.376480:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565601408:192.168.203.147@tcp:4:dd.0 00000100:00100000:1.0:1789760796.390913:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985825 gen 3 00000008:00010000:1.0:1789760796.390948:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:1.0:1789760796.390955:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:1.0:1789760796.390963:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:1.0:1789760796.390984:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:1.0:1789760796.391008:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565601408:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.391106:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18517, 18517] 00000080:00200000:2.0:1789760796.391119:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.391123:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.391135:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75849728 i_blocks 148168 00000020:00200000:2.0:1789760796.391138:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.391146:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.391172:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75849728, count: 4096 00000080:00200000:2.0:1789760796.391179:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75849728, bytes: 4096 00000080:00200000:2.0:1789760796.391187:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.391195:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.391197:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75849728, 75853824) 0 0 00000080:00200000:2.0:1789760796.391201:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.391216:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18518, 18518] 00020000:00200000:2.0:1789760796.391223:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75849728 chunk: [75849728, 75853824) 75853824, 4096 00020000:00200000:2.0:1789760796.391227:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.391242:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18518, 18518] 00000008:00010000:2.0:1789760796.391257:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.391262:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.391272:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75849728, 75853824) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.391284:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75849728=0x4856000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.391331:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.391334:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75853824 00000020:00010000:2.0:1789760796.391341:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.391344:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.391347:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.391349:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.391352:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.391358:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.391363:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.391388:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760796.391482:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0380 - object 0x240000400:5 offset 75849728<>75853824 00000100:00100000:3.0:1789760796.391577:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565601536:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.404732:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985826 gen 3 00000008:00010000:3.0:1789760796.404778:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760796.404784:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.404794:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.404817:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760796.404843:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565601536:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.405045:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18518, 18518] 00000080:00200000:2.0:1789760796.405057:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.405062:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.405074:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75853824 i_blocks 148176 00000020:00200000:2.0:1789760796.405078:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.405087:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.405112:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75853824, count: 4096 00000080:00200000:2.0:1789760796.405119:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75853824, bytes: 4096 00000080:00200000:2.0:1789760796.405126:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.405133:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.405135:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75853824, 75857920) 0 0 00000080:00200000:2.0:1789760796.405139:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.405151:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18519, 18519] 00020000:00200000:2.0:1789760796.405158:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75853824 chunk: [75853824, 75857920) 75857920, 4096 00020000:00200000:2.0:1789760796.405162:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.405176:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18519, 18519] 00000008:00010000:2.0:1789760796.405189:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.405193:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.405202:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75853824, 75857920) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.405213:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75853824=0x4857000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.405258:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.405261:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75857920 00000020:00010000:2.0:1789760796.405268:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.405271:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.405274:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.405277:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.405281:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.405288:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.405294:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.405308:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760796.405423:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1500 - object 0x240000400:5 offset 75853824<>75857920 00000100:00100000:3.0:1789760796.405510:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760796.405584:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565601664:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.416090:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985827 gen 3 00000008:00010000:3.0:1789760796.416126:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760796.416133:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.416142:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.416250:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760796.416276:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565601664:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.416923:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18519, 18519] 00000080:00200000:2.0:1789760796.416937:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.416942:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.416955:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75857920 i_blocks 148184 00000020:00200000:2.0:1789760796.416958:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.416967:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.416994:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75857920, count: 4096 00000080:00200000:2.0:1789760796.417000:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75857920, bytes: 4096 00000080:00200000:2.0:1789760796.417008:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.417015:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.417017:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75857920, 75862016) 0 0 00000080:00200000:2.0:1789760796.417021:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.417035:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18520, 18520] 00020000:00200000:2.0:1789760796.417042:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75857920 chunk: [75857920, 75862016) 75862016, 4096 00020000:00200000:2.0:1789760796.417046:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.417061:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18520, 18520] 00000008:00010000:2.0:1789760796.417074:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.417079:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.417087:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75857920, 75862016) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.417098:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75857920=0x4858000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.417140:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.417143:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75862016 00000020:00010000:2.0:1789760796.417149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.417152:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.417154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.417157:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.417160:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.417166:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.417171:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.417187:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760796.417281:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1180 - object 0x240000400:5 offset 75857920<>75862016 00000100:00100000:3.0:1789760796.418626:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565601792:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.429788:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985828 gen 3 00000008:00010000:3.0:1789760796.429823:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:3.0:1789760796.429830:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.429837:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.429857:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:3.0:1789760796.429882:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565601792:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.429887:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18520, 18520] 00000080:00200000:2.0:1789760796.429901:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.429906:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.429920:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75862016 i_blocks 148192 00000020:00200000:2.0:1789760796.429924:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.429933:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.429956:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75862016, count: 4096 00000080:00200000:2.0:1789760796.429962:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75862016, bytes: 4096 00000080:00200000:2.0:1789760796.429970:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.429977:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.429979:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75862016, 75866112) 0 0 00000080:00200000:2.0:1789760796.429983:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.429996:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18521, 18521] 00020000:00200000:2.0:1789760796.430002:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75862016 chunk: [75862016, 75866112) 75866112, 4096 00020000:00200000:2.0:1789760796.430006:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.430020:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18521, 18521] 00000008:00010000:2.0:1789760796.430033:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.430038:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.430047:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75862016, 75866112) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.430058:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75862016=0x4859000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.430102:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.430105:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75866112 00000020:00010000:2.0:1789760796.430112:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.430116:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.430118:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.430121:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.430124:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.430130:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.430136:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.430150:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760796.430243:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2680 - object 0x240000400:5 offset 75862016<>75866112 00000100:00100000:3.0:1789760796.430347:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760796.430362:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565601920:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.439213:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985829 gen 3 00000008:00010000:3.0:1789760796.439245:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:3.0:1789760796.439250:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.439257:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.439307:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:3.0:1789760796.439332:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565601920:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.439406:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18521, 18521] 00000080:00200000:2.0:1789760796.439419:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.439424:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.439436:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75866112 i_blocks 148200 00000020:00200000:2.0:1789760796.439439:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.439448:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.439474:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75866112, count: 4096 00000080:00200000:2.0:1789760796.439481:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75866112, bytes: 4096 00000080:00200000:2.0:1789760796.439549:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.439557:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.439559:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75866112, 75870208) 0 0 00000080:00200000:2.0:1789760796.439563:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.439576:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18522, 18522] 00020000:00200000:2.0:1789760796.439582:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75866112 chunk: [75866112, 75870208) 75870208, 4096 00020000:00200000:2.0:1789760796.439586:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.439599:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18522, 18522] 00000008:00010000:2.0:1789760796.439745:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.439751:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.439760:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75866112, 75870208) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.439770:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75866112=0x485a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.439808:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.439811:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75870208 00000020:00010000:2.0:1789760796.439817:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.439820:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.439823:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.439826:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.439829:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.439835:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.439840:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.439853:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760796.439937:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0a80 - object 0x240000400:5 offset 75866112<>75870208 00000100:00100000:3.0:1789760796.440034:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565602048:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.451831:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985830 gen 3 00000008:00010000:3.0:1789760796.451870:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760796.451877:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.451885:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.451906:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760796.451932:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565602048:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.452032:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18522, 18522] 00000080:00200000:2.0:1789760796.452045:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.452050:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.452061:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75870208 i_blocks 148208 00000020:00200000:2.0:1789760796.452064:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.452073:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.452099:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75870208, count: 4096 00000080:00200000:2.0:1789760796.452105:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75870208, bytes: 4096 00000080:00200000:2.0:1789760796.452112:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.452119:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.452121:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75870208, 75874304) 0 0 00000080:00200000:2.0:1789760796.452125:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.452138:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18523, 18523] 00020000:00200000:2.0:1789760796.452145:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75870208 chunk: [75870208, 75874304) 75874304, 4096 00020000:00200000:2.0:1789760796.452148:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.452164:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18523, 18523] 00000008:00010000:2.0:1789760796.452176:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.452181:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.452191:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75870208, 75874304) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.452201:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75870208=0x485b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.452244:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.452247:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75874304 00000020:00010000:2.0:1789760796.452253:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.452257:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.452259:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.452262:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.452265:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.452272:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.452277:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.452290:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760796.452402:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3100 - object 0x240000400:5 offset 75870208<>75874304 00000100:00100000:3.0:1789760796.452626:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760796.452640:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565602176:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.468344:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985831 gen 3 00000008:00010000:3.0:1789760796.468392:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760796.468399:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.468408:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.468482:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760796.468506:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565602176:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.468626:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18523, 18523] 00000080:00200000:2.0:1789760796.468638:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.468669:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.468682:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75874304 i_blocks 148216 00000020:00200000:2.0:1789760796.468686:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.468694:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.468718:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75874304, count: 4096 00000080:00200000:2.0:1789760796.468724:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75874304, bytes: 4096 00000080:00200000:2.0:1789760796.468731:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.468739:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.468741:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75874304, 75878400) 0 0 00000080:00200000:2.0:1789760796.468745:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.468758:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18524, 18524] 00020000:00200000:2.0:1789760796.468765:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75874304 chunk: [75874304, 75878400) 75878400, 4096 00020000:00200000:2.0:1789760796.468769:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.468784:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18524, 18524] 00000008:00010000:2.0:1789760796.468797:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.468802:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.468811:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75874304, 75878400) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.468823:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75874304=0x485c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.468863:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.468865:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75878400 00000020:00010000:2.0:1789760796.468872:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.468875:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.468878:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.468882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.468885:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.468892:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.468898:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.468911:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760796.468998:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 75874304<>75878400 00000100:00100000:3.0:1789760796.469054:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565602304:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760796.485069:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985832 gen 3 00000008:00010000:2.0:1789760796.485295:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760796.485303:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760796.485383:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760796.485466:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760796.485489:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565602304:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.485566:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18524, 18524] 00000080:00200000:1.0:1789760796.485579:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.485584:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.485595:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75878400 i_blocks 148224 00000020:00200000:1.0:1789760796.485599:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.485607:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.485637:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75878400, count: 4096 00000080:00200000:1.0:1789760796.485664:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75878400, bytes: 4096 00000080:00200000:1.0:1789760796.485675:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.485682:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.485685:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75878400, 75882496) 0 0 00000080:00200000:1.0:1789760796.485688:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.485702:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18525, 18525] 00020000:00200000:1.0:1789760796.485708:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75878400 chunk: [75878400, 75882496) 75882496, 4096 00020000:00200000:1.0:1789760796.485713:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.485797:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18525, 18525] 00000008:00010000:1.0:1789760796.485870:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.485876:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.485888:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75878400, 75882496) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.485903:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75878400=0x485d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.486017:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.486020:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75882496 00000020:00010000:1.0:1789760796.486028:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.486031:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.486033:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.486036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.486039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.486045:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.486050:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.486063:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760796.486283:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 75878400<>75882496 00000100:00100000:0.0:1789760796.486368:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565602432:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.500386:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985833 gen 3 00000008:00010000:0.0:1789760796.500426:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760796.500431:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.500448:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.500471:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760796.500494:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565602432:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.501220:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18525, 18525] 00000080:00200000:1.0:1789760796.501235:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.501240:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.501252:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75882496 i_blocks 148232 00000020:00200000:1.0:1789760796.501256:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.501264:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.501289:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75882496, count: 4096 00000080:00200000:1.0:1789760796.501295:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75882496, bytes: 4096 00000080:00200000:1.0:1789760796.501306:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.501388:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.501390:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75882496, 75886592) 0 0 00000080:00200000:1.0:1789760796.501394:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.501408:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18526, 18526] 00020000:00200000:1.0:1789760796.501415:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75882496 chunk: [75882496, 75886592) 75886592, 4096 00020000:00200000:1.0:1789760796.501419:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.501436:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18526, 18526] 00000008:00010000:1.0:1789760796.501449:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.501454:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.501463:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75882496, 75886592) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.501474:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75882496=0x485e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.501522:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.501525:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75886592 00000020:00010000:1.0:1789760796.501532:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.501535:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.501538:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.501541:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.501544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.501550:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.501555:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.501571:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760796.501708:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 75882496<>75886592 00000100:00100000:0.0:1789760796.502855:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565602560:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.518098:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985834 gen 3 00000008:00010000:0.0:1789760796.518132:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760796.518138:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.518147:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.518175:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760796.518197:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565602560:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.518431:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18526, 18526] 00000080:00200000:1.0:1789760796.518443:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.518449:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.518460:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75886592 i_blocks 148240 00000020:00200000:1.0:1789760796.518463:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.518471:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.518495:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75886592, count: 4096 00000080:00200000:1.0:1789760796.518501:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75886592, bytes: 4096 00000080:00200000:1.0:1789760796.518510:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.518517:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.518519:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75886592, 75890688) 0 0 00000080:00200000:1.0:1789760796.518523:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.518537:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18527, 18527] 00020000:00200000:1.0:1789760796.518543:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75886592 chunk: [75886592, 75890688) 75890688, 4096 00020000:00200000:1.0:1789760796.518547:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.518562:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18527, 18527] 00000008:00010000:1.0:1789760796.518574:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.518579:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.518588:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75886592, 75890688) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.518598:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75886592=0x485f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.518641:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.518673:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75890688 00000020:00010000:1.0:1789760796.518680:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.518683:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.518686:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.518689:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.518692:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.518698:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.518704:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.518716:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760796.518808:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 75886592<>75890688 00000100:00100000:0.0:1789760796.518955:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565602688:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.532472:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985835 gen 3 00000008:00010000:0.0:1789760796.532520:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760796.532528:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.532537:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.532557:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760796.532584:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565602688:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.532742:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18527, 18527] 00000080:00200000:1.0:1789760796.532756:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.532761:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.532774:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75890688 i_blocks 148248 00000020:00200000:1.0:1789760796.532777:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.532785:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.532814:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75890688, count: 4096 00000080:00200000:1.0:1789760796.532820:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75890688, bytes: 4096 00000080:00200000:1.0:1789760796.532828:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.532835:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.532838:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75890688, 75894784) 0 0 00000080:00200000:1.0:1789760796.532841:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.532853:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18528, 18528] 00020000:00200000:1.0:1789760796.532860:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75890688 chunk: [75890688, 75894784) 75894784, 4096 00020000:00200000:1.0:1789760796.532864:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.532879:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18528, 18528] 00000008:00010000:1.0:1789760796.532892:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.532898:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.532907:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75890688, 75894784) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.532918:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75890688=0x4860000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.532963:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.532966:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75894784 00000020:00010000:1.0:1789760796.532972:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.532975:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.532978:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.532982:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.532984:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.532991:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.532996:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.533009:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760796.533105:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 75890688<>75894784 00000100:00100000:0.0:1789760796.533241:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565602816:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.546745:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985836 gen 3 00000008:00010000:0.0:1789760796.546787:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760796.546794:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.546803:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.546822:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760796.546846:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565602816:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.547066:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18528, 18528] 00000080:00200000:1.0:1789760796.547079:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.547084:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.547096:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75894784 i_blocks 148256 00000020:00200000:1.0:1789760796.547099:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.547107:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.547134:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75894784, count: 4096 00000080:00200000:1.0:1789760796.547140:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75894784, bytes: 4096 00000080:00200000:1.0:1789760796.547149:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.547156:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.547159:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75894784, 75898880) 0 0 00000080:00200000:1.0:1789760796.547163:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.547177:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18529, 18529] 00020000:00200000:1.0:1789760796.547184:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75894784 chunk: [75894784, 75898880) 75898880, 4096 00020000:00200000:1.0:1789760796.547189:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.547205:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18529, 18529] 00000008:00010000:1.0:1789760796.547219:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.547225:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.547236:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75894784, 75898880) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.547247:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75894784=0x4861000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.547289:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.547292:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75898880 00000020:00010000:1.0:1789760796.547298:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.547300:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.547303:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.547307:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.547310:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.547316:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.547321:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.547335:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760796.547445:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 75894784<>75898880 00000100:00100000:0.0:1789760796.548782:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565602944:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.562831:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985837 gen 3 00000008:00010000:0.0:1789760796.562870:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760796.562876:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.562885:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.562986:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760796.563012:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565602944:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.563100:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18529, 18529] 00000080:00200000:1.0:1789760796.563115:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.563121:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.563134:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75898880 i_blocks 148264 00000020:00200000:1.0:1789760796.563138:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.563147:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.563173:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75898880, count: 4096 00000080:00200000:1.0:1789760796.563180:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75898880, bytes: 4096 00000080:00200000:1.0:1789760796.563191:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.563198:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.563200:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75898880, 75902976) 0 0 00000080:00200000:1.0:1789760796.563204:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.563217:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18530, 18530] 00020000:00200000:1.0:1789760796.563225:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75898880 chunk: [75898880, 75902976) 75902976, 4096 00020000:00200000:1.0:1789760796.563229:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.563244:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18530, 18530] 00000008:00010000:1.0:1789760796.563257:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.563262:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.563271:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75898880, 75902976) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.563281:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75898880=0x4862000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.563324:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.563326:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75902976 00000020:00010000:1.0:1789760796.563333:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.563336:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.563339:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.563342:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.563345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.563350:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.563356:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.563369:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760796.564036:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 75898880<>75902976 00000100:00100000:0.0:1789760796.564093:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565603072:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.576962:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985838 gen 3 00000008:00010000:0.0:1789760796.577003:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760796.577010:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.577019:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.577043:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760796.577068:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565603072:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.577589:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18530, 18530] 00000080:00200000:2.0:1789760796.577606:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.577611:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.577624:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75902976 i_blocks 148272 00000020:00200000:2.0:1789760796.577627:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.577637:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.577700:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75902976, count: 4096 00000080:00200000:2.0:1789760796.577707:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75902976, bytes: 4096 00000080:00200000:2.0:1789760796.577715:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.577723:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.577725:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75902976, 75907072) 0 0 00000080:00200000:2.0:1789760796.577729:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.577741:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18531, 18531] 00020000:00200000:2.0:1789760796.577748:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75902976 chunk: [75902976, 75907072) 75907072, 4096 00020000:00200000:2.0:1789760796.577752:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.577767:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18531, 18531] 00000008:00010000:2.0:1789760796.577780:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.577785:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.577794:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75902976, 75907072) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.577805:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75902976=0x4863000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.577860:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.577863:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75907072 00000020:00010000:2.0:1789760796.577870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.577873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.577876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.577879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e218 (ffff93f84354e008/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.577882:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e008/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.577889:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.577894:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.577909:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760796.578028:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 75902976<>75907072 00000100:00100000:2.0:1789760796.578076:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760796.578094:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565603200:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760796.594888:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985839 gen 3 00000008:00010000:2.0:1789760796.594927:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:2.0:1789760796.594944:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760796.594954:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760796.594980:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:2.0:1789760796.595008:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565603200:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760796.595160:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18531, 18531] 00000080:00200000:3.0:1789760796.595184:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760796.595190:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760796.595201:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75907072 i_blocks 148280 00000020:00200000:3.0:1789760796.595205:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760796.595213:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760796.595265:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75907072, count: 4096 00000080:00200000:3.0:1789760796.595272:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75907072, bytes: 4096 00000080:00200000:3.0:1789760796.595283:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760796.595291:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760796.595293:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75907072, 75911168) 0 0 00000080:00200000:3.0:1789760796.595297:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760796.595311:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18532, 18532] 00020000:00200000:3.0:1789760796.595317:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75907072 chunk: [75907072, 75911168) 75911168, 4096 00020000:00200000:3.0:1789760796.595321:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760796.595337:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18532, 18532] 00000008:00010000:3.0:1789760796.595350:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760796.595354:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760796.595363:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75907072, 75911168) append 0 append lockpos 0 00000080:00200000:3.0:1789760796.595386:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75907072=0x4864000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760796.595430:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760796.595433:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75911168 00000020:00010000:3.0:1789760796.595440:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760796.595443:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.595446:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.595449:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760796.595451:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760796.595458:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760796.595464:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760796.595477:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760796.595574:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 75907072<>75911168 00000100:00100000:3.0:1789760796.595610:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760796.595623:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565603328:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.617927:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985840 gen 3 00000008:00010000:3.0:1789760796.617962:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:3.0:1789760796.617968:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.617977:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.617992:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:3.0:1789760796.618014:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565603328:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.618406:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18532, 18532] 00000080:00200000:0.0:1789760796.618420:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.618425:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.618438:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75911168 i_blocks 148288 00000020:00200000:0.0:1789760796.618441:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.618449:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.618474:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75911168, count: 4096 00000080:00200000:0.0:1789760796.618480:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75911168, bytes: 4096 00000080:00200000:0.0:1789760796.618487:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.618495:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.618497:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75911168, 75915264) 0 0 00000080:00200000:0.0:1789760796.618945:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.618959:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18533, 18533] 00020000:00200000:0.0:1789760796.618967:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75911168 chunk: [75911168, 75915264) 75915264, 4096 00020000:00200000:0.0:1789760796.618971:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.618988:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18533, 18533] 00000008:00010000:0.0:1789760796.619001:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.619006:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.619014:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75911168, 75915264) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.619025:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75911168=0x4865000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.619071:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.619074:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75915264 00000020:00010000:0.0:1789760796.619080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.619083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.619086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.619090:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.619092:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.619098:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.619103:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.619119:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760796.619219:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 75911168<>75915264 00000100:00100000:1.0:1789760796.619289:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565603456:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.630629:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985841 gen 3 00000008:00010000:0.0:1789760796.630689:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760796.630695:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.630704:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.630725:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760796.630752:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565603456:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760796.632559:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18533, 18533] 00000080:00200000:3.0:1789760796.632577:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760796.632582:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760796.632596:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75915264 i_blocks 148296 00000020:00200000:3.0:1789760796.632600:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760796.632614:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760796.632639:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75915264, count: 4096 00000080:00200000:3.0:1789760796.632685:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75915264, bytes: 4096 00000080:00200000:3.0:1789760796.632695:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760796.632703:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760796.632705:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75915264, 75919360) 0 0 00000080:00200000:3.0:1789760796.632709:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760796.632721:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18534, 18534] 00020000:00200000:3.0:1789760796.632729:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75915264 chunk: [75915264, 75919360) 75919360, 4096 00020000:00200000:3.0:1789760796.632732:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760796.632747:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18534, 18534] 00000008:00010000:3.0:1789760796.632760:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760796.632765:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760796.632773:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75915264, 75919360) append 0 append lockpos 0 00000080:00200000:3.0:1789760796.632783:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75915264=0x4866000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760796.632840:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760796.632843:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75919360 00000020:00010000:3.0:1789760796.632850:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760796.632853:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.632855:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.632858:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760796.632860:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760796.632867:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760796.632877:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760796.632894:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760796.632991:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66a00 - object 0x240000400:5 offset 75915264<>75919360 00000100:00100000:2.0:1789760796.633072:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565603584:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760796.643687:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985842 gen 3 00000008:00010000:2.0:1789760796.643733:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760796.643740:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760796.643750:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760796.643767:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760796.643789:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565603584:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760796.643955:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18534, 18534] 00000080:00200000:3.0:1789760796.643968:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760796.643972:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760796.643985:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75919360 i_blocks 148304 00000020:00200000:3.0:1789760796.643989:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760796.643997:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760796.644024:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75919360, count: 4096 00000080:00200000:3.0:1789760796.644030:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75919360, bytes: 4096 00000080:00200000:3.0:1789760796.644038:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760796.644045:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760796.644047:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75919360, 75923456) 0 0 00000080:00200000:3.0:1789760796.644051:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760796.644064:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18535, 18535] 00020000:00200000:3.0:1789760796.644071:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75919360 chunk: [75919360, 75923456) 75923456, 4096 00020000:00200000:3.0:1789760796.644075:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760796.644169:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18535, 18535] 00000008:00010000:3.0:1789760796.644184:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760796.644188:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760796.644198:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75919360, 75923456) append 0 append lockpos 0 00000080:00200000:3.0:1789760796.644209:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75919360=0x4867000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760796.644254:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760796.644257:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75923456 00000020:00010000:3.0:1789760796.644264:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760796.644267:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.644270:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.644272:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760796.644275:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760796.644280:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760796.644285:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760796.644298:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3620c0 00000008:00100000:3.0:1789760796.644461:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65c00 - object 0x240000400:5 offset 75919360<>75923456 00000100:00100000:2.0:1789760796.646886:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565603712:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760796.657695:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985843 gen 3 00000008:00010000:2.0:1789760796.657743:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3620c0 00000020:00200000:2.0:1789760796.657750:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760796.657759:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760796.657773:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3620c0 00000100:00100000:2.0:1789760796.657794:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565603712:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760796.658132:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18535, 18535] 00000080:00200000:3.0:1789760796.658144:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760796.658149:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760796.658160:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75923456 i_blocks 148312 00000020:00200000:3.0:1789760796.658164:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760796.658172:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760796.658199:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75923456, count: 4096 00000080:00200000:3.0:1789760796.658206:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75923456, bytes: 4096 00000080:00200000:3.0:1789760796.658212:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760796.658219:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760796.658222:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75923456, 75927552) 0 0 00000080:00200000:3.0:1789760796.658226:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760796.658237:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18536, 18536] 00020000:00200000:3.0:1789760796.658244:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75923456 chunk: [75923456, 75927552) 75927552, 4096 00020000:00200000:3.0:1789760796.658248:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760796.658262:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18536, 18536] 00000008:00010000:3.0:1789760796.658275:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760796.658280:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760796.658458:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75923456, 75927552) append 0 append lockpos 0 00000080:00200000:3.0:1789760796.658471:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75923456=0x4868000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760796.658528:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760796.658531:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75927552 00000020:00010000:3.0:1789760796.658538:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760796.658541:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.658544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.658548:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760796.658551:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760796.658558:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760796.658563:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760796.658578:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362600 00000008:00100000:3.0:1789760796.658704:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64000 - object 0x240000400:5 offset 75923456<>75927552 00000100:00100000:2.0:1789760796.658873:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565603840:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760796.669857:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985844 gen 3 00000008:00010000:2.0:1789760796.669892:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362600 00000020:00200000:2.0:1789760796.669898:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760796.669907:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760796.669975:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362600 00000100:00100000:2.0:1789760796.669999:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565603840:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760796.670768:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18536, 18536] 00000080:00200000:3.0:1789760796.670781:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760796.670786:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760796.670799:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75927552 i_blocks 148320 00000020:00200000:3.0:1789760796.670803:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760796.670811:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760796.670836:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75927552, count: 4096 00000080:00200000:3.0:1789760796.670842:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75927552, bytes: 4096 00000080:00200000:3.0:1789760796.670850:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760796.670858:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760796.670860:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75927552, 75931648) 0 0 00000080:00200000:3.0:1789760796.670863:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760796.670876:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18537, 18537] 00020000:00200000:3.0:1789760796.670883:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75927552 chunk: [75927552, 75931648) 75931648, 4096 00020000:00200000:3.0:1789760796.670887:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760796.670902:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18537, 18537] 00000008:00010000:3.0:1789760796.670914:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760796.670919:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760796.670927:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75927552, 75931648) append 0 append lockpos 0 00000080:00200000:3.0:1789760796.670937:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75927552=0x4869000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760796.670984:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760796.670986:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75931648 00000020:00010000:3.0:1789760796.670992:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760796.670995:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.670998:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.671001:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760796.671004:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760796.671010:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760796.671016:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760796.671032:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362900 00000008:00100000:3.0:1789760796.671132:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64e00 - object 0x240000400:5 offset 75927552<>75931648 00000100:00100000:2.0:1789760796.671421:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565603968:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760796.684548:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985845 gen 3 00000008:00010000:2.0:1789760796.684605:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362900 00000020:00200000:2.0:1789760796.684614:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760796.684628:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760796.684690:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362900 00000100:00100000:2.0:1789760796.684713:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565603968:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760796.684897:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18537, 18537] 00000080:00200000:3.0:1789760796.684910:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760796.684914:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760796.684926:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75931648 i_blocks 148328 00000020:00200000:3.0:1789760796.684930:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760796.684939:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760796.684962:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75931648, count: 4096 00000080:00200000:3.0:1789760796.684968:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75931648, bytes: 4096 00000080:00200000:3.0:1789760796.684977:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760796.684985:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760796.684987:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75931648, 75935744) 0 0 00000080:00200000:3.0:1789760796.684991:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760796.685004:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18538, 18538] 00020000:00200000:3.0:1789760796.685011:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75931648 chunk: [75931648, 75935744) 75935744, 4096 00020000:00200000:3.0:1789760796.685015:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760796.685029:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18538, 18538] 00000008:00010000:3.0:1789760796.685042:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760796.685046:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760796.685054:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75931648, 75935744) append 0 append lockpos 0 00000080:00200000:3.0:1789760796.685064:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75931648=0x486a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760796.685179:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760796.685183:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75935744 00000020:00010000:3.0:1789760796.685190:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760796.685193:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.685196:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760796.685199:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760796.685202:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760796.685208:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760796.685213:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760796.685225:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760796.685396:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 75931648<>75935744 00000100:00100000:3.0:1789760796.686017:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565604096:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.705244:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985846 gen 3 00000008:00010000:3.0:1789760796.705277:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:3.0:1789760796.705284:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.705291:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.705370:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:3.0:1789760796.705402:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565604096:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760796.706027:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18538, 18538] 00000080:00200000:0.0:1789760796.706041:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760796.706046:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760796.706211:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75935744 i_blocks 148336 00000020:00200000:0.0:1789760796.706215:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760796.706224:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760796.706249:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75935744, count: 4096 00000080:00200000:0.0:1789760796.706255:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75935744, bytes: 4096 00000080:00200000:0.0:1789760796.706264:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760796.706271:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760796.706273:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75935744, 75939840) 0 0 00000080:00200000:0.0:1789760796.706276:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760796.706287:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18539, 18539] 00020000:00200000:0.0:1789760796.706294:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75935744 chunk: [75935744, 75939840) 75939840, 4096 00020000:00200000:0.0:1789760796.706297:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760796.706484:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18539, 18539] 00000008:00010000:0.0:1789760796.706565:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760796.706571:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760796.706581:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75935744, 75939840) append 0 append lockpos 0 00000080:00200000:0.0:1789760796.706593:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75935744=0x486b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760796.706637:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760796.706640:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75939840 00000020:00010000:0.0:1789760796.706710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760796.706715:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.706784:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760796.706789:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760796.706791:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760796.706799:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760796.706805:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.706819:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760796.706980:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 75935744<>75939840 00000100:00100000:0.0:1789760796.707023:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565604224:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.726701:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985847 gen 3 00000008:00010000:0.0:1789760796.726742:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760796.726750:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.726758:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.726774:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760796.726798:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565604224:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.726971:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18539, 18539] 00000080:00200000:1.0:1789760796.726983:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.726987:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.726998:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75939840 i_blocks 148344 00000020:00200000:1.0:1789760796.727001:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.727009:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.727034:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75939840, count: 4096 00000080:00200000:1.0:1789760796.727039:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75939840, bytes: 4096 00000080:00200000:1.0:1789760796.727046:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.727054:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.727056:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75939840, 75943936) 0 0 00000080:00200000:1.0:1789760796.727060:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.727072:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18540, 18540] 00020000:00200000:1.0:1789760796.727078:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75939840 chunk: [75939840, 75943936) 75943936, 4096 00020000:00200000:1.0:1789760796.727082:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.727162:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18540, 18540] 00000008:00010000:1.0:1789760796.727175:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.727180:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.727188:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75939840, 75943936) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.727198:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75939840=0x486c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.727237:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.727240:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75943936 00000020:00010000:1.0:1789760796.727245:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.727248:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.727250:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.727253:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.727256:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.727261:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.727266:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.727279:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760796.727382:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 75939840<>75943936 00000100:00100000:0.0:1789760796.727539:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565604352:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.746699:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985848 gen 3 00000008:00010000:0.0:1789760796.746749:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760796.746755:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.746765:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.746774:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760796.746798:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565604352:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.746932:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18540, 18540] 00000080:00200000:1.0:1789760796.746946:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.746951:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.746963:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75943936 i_blocks 148352 00000020:00200000:1.0:1789760796.746967:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.746974:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.746999:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75943936, count: 4096 00000080:00200000:1.0:1789760796.747006:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75943936, bytes: 4096 00000080:00200000:1.0:1789760796.747013:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.747021:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.747023:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75943936, 75948032) 0 0 00000080:00200000:1.0:1789760796.747027:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.747040:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18541, 18541] 00020000:00200000:1.0:1789760796.747046:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75943936 chunk: [75943936, 75948032) 75948032, 4096 00020000:00200000:1.0:1789760796.747049:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.747064:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18541, 18541] 00000008:00010000:1.0:1789760796.747077:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.747081:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.747475:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75943936, 75948032) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.747497:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75943936=0x486d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.747689:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.747693:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75948032 00000020:00010000:1.0:1789760796.747701:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.747704:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.747707:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.747711:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.747713:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.747812:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.747819:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.747896:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760796.748002:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 75943936<>75948032 00000100:00100000:0.0:1789760796.748057:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565604480:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.759270:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985849 gen 3 00000008:00010000:0.0:1789760796.759307:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760796.759440:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.759450:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.759469:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000080:00200000:1.0:1789760796.759574:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18541, 18541] 00000080:00200000:1.0:1789760796.759591:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.759595:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.759611:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75948032 i_blocks 148360 00000020:00200000:1.0:1789760796.759614:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.759622:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.759683:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75948032, count: 4096 00000080:00200000:1.0:1789760796.759690:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75948032, bytes: 4096 00000080:00200000:1.0:1789760796.759698:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.759705:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.759707:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75948032, 75952128) 0 0 00000080:00200000:1.0:1789760796.759710:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.759734:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18542, 18542] 00020000:00200000:1.0:1789760796.759741:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75948032 chunk: [75948032, 75952128) 75952128, 4096 00020000:00200000:1.0:1789760796.759745:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.759761:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18542, 18542] 00000008:00010000:1.0:1789760796.759776:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.759780:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.759790:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75948032, 75952128) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.759800:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75948032=0x486e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.760042:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.760046:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75952128 00000020:00010000:1.0:1789760796.760054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.760057:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.760060:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.760064:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.760066:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.760073:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.760078:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:0.0:1789760796.760144:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565604480:192.168.203.147@tcp:4:dd.0 00000008:00010000:0.0:1789760796.760183:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:0.0:1789760796.760282:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 75948032<>75952128 00000100:00100000:0.0:1789760796.760470:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760796.760568:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565604608:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.776330:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985850 gen 3 00000008:00010000:0.0:1789760796.776368:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760796.776382:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.776391:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.776409:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760796.776432:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565604608:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.776522:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18542, 18542] 00000080:00200000:1.0:1789760796.776535:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.776540:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.776552:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75952128 i_blocks 148368 00000020:00200000:1.0:1789760796.776555:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.776563:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.776589:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75952128, count: 4096 00000080:00200000:1.0:1789760796.776595:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75952128, bytes: 4096 00000080:00200000:1.0:1789760796.776603:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.776610:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.776611:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75952128, 75956224) 0 0 00000080:00200000:1.0:1789760796.776615:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.776627:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18543, 18543] 00020000:00200000:1.0:1789760796.776634:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75952128 chunk: [75952128, 75956224) 75956224, 4096 00020000:00200000:1.0:1789760796.776638:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.776687:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18543, 18543] 00000008:00010000:1.0:1789760796.776701:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.776707:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.776726:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75952128, 75956224) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.776738:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75952128=0x486f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.776790:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.776792:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75956224 00000020:00010000:1.0:1789760796.776799:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.776803:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.776806:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.776809:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.776812:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.776818:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.776823:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.777007:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760796.777105:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 75952128<>75956224 00000100:00100000:1.0:1789760796.777136:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760796.777151:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565604736:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.795347:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985851 gen 3 00000008:00010000:0.0:1789760796.795415:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760796.795422:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.795431:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.795452:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760796.795476:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565604736:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.795626:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18543, 18543] 00000080:00200000:1.0:1789760796.795641:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.795679:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.795692:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75956224 i_blocks 148376 00000020:00200000:1.0:1789760796.795696:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.795704:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.795731:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75956224, count: 4096 00000080:00200000:1.0:1789760796.795737:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75956224, bytes: 4096 00000080:00200000:1.0:1789760796.795760:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.795767:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.795770:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75956224, 75960320) 0 0 00000080:00200000:1.0:1789760796.795774:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.795788:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18544, 18544] 00020000:00200000:1.0:1789760796.795794:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75956224 chunk: [75956224, 75960320) 75960320, 4096 00020000:00200000:1.0:1789760796.795798:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.795814:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18544, 18544] 00000008:00010000:1.0:1789760796.795828:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.795843:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.795855:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75956224, 75960320) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.795867:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75956224=0x4870000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.795909:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.795911:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75960320 00000020:00010000:1.0:1789760796.795918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.795921:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.795923:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.795926:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.795929:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.795935:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.795940:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.795953:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760796.796043:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 75956224<>75960320 00000100:00100000:0.0:1789760796.796282:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565604864:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.809892:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985852 gen 3 00000008:00010000:0.0:1789760796.809924:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760796.809929:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.809937:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.809954:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760796.809975:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565604864:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.810115:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18544, 18544] 00000080:00200000:1.0:1789760796.810130:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.810135:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.810148:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75960320 i_blocks 148384 00000020:00200000:1.0:1789760796.810152:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.810159:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.810186:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75960320, count: 4096 00000080:00200000:1.0:1789760796.810193:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75960320, bytes: 4096 00000080:00200000:1.0:1789760796.810201:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.810208:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.810210:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75960320, 75964416) 0 0 00000080:00200000:1.0:1789760796.810214:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.810228:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18545, 18545] 00020000:00200000:1.0:1789760796.810236:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75960320 chunk: [75960320, 75964416) 75964416, 4096 00020000:00200000:1.0:1789760796.810240:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.810255:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18545, 18545] 00000008:00010000:1.0:1789760796.810269:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.810273:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.810282:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75960320, 75964416) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.810293:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75960320=0x4871000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.810333:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.810336:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75964416 00000020:00010000:1.0:1789760796.810342:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.810345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.810348:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.810351:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.810354:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.810360:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.810365:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.810388:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760796.810486:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 75960320<>75964416 00000100:00100000:0.0:1789760796.810689:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760796.810704:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565604992:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.821507:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985853 gen 3 00000008:00010000:0.0:1789760796.821544:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760796.821552:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.821560:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.821578:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760796.821602:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565604992:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.821712:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18545, 18545] 00000080:00200000:1.0:1789760796.821725:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.821730:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.821741:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75964416 i_blocks 148392 00000020:00200000:1.0:1789760796.821745:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.821753:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.821778:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75964416, count: 4096 00000080:00200000:1.0:1789760796.821784:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75964416, bytes: 4096 00000080:00200000:1.0:1789760796.821792:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.821800:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.821802:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75964416, 75968512) 0 0 00000080:00200000:1.0:1789760796.821807:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.821819:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18546, 18546] 00020000:00200000:1.0:1789760796.821827:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75964416 chunk: [75964416, 75968512) 75968512, 4096 00020000:00200000:1.0:1789760796.821830:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.821845:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18546, 18546] 00000008:00010000:1.0:1789760796.821859:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.821864:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.821872:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75964416, 75968512) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.821883:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75964416=0x4872000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.821932:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.821935:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75968512 00000020:00010000:1.0:1789760796.821942:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.821945:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.821948:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.821951:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.821953:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.821960:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.821966:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.821980:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760796.822075:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 75964416<>75968512 00000100:00100000:0.0:1789760796.822179:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565605120:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.841519:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985854 gen 3 00000008:00010000:0.0:1789760796.841555:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760796.841562:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.841570:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.841589:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760796.841614:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565605120:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.841735:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18546, 18546] 00000080:00200000:1.0:1789760796.841748:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.841753:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.841765:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75968512 i_blocks 148400 00000020:00200000:1.0:1789760796.841768:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.841779:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.841804:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75968512, count: 4096 00000080:00200000:1.0:1789760796.841810:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75968512, bytes: 4096 00000080:00200000:1.0:1789760796.841819:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.841826:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.841828:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75968512, 75972608) 0 0 00000080:00200000:1.0:1789760796.841831:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.841844:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18547, 18547] 00020000:00200000:1.0:1789760796.841851:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75968512 chunk: [75968512, 75972608) 75972608, 4096 00020000:00200000:1.0:1789760796.841855:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.841869:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18547, 18547] 00000008:00010000:1.0:1789760796.841883:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.841887:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.841894:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75968512, 75972608) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.841904:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75968512=0x4873000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.841946:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.841949:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75972608 00000020:00010000:1.0:1789760796.841956:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.841959:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.841962:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.841965:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.841967:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.841972:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.841977:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.841992:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760796.842088:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 75968512<>75972608 00000100:00100000:0.0:1789760796.842298:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565605248:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.858838:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985855 gen 3 00000008:00010000:0.0:1789760796.858878:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760796.858884:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.858892:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.858911:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760796.858932:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565605248:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.859036:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18547, 18547] 00000080:00200000:1.0:1789760796.859050:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.859055:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.859069:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75972608 i_blocks 148408 00000020:00200000:1.0:1789760796.859072:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.859081:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.859107:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75972608, count: 4096 00000080:00200000:1.0:1789760796.859113:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75972608, bytes: 4096 00000080:00200000:1.0:1789760796.859122:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.859129:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.859131:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75972608, 75976704) 0 0 00000080:00200000:1.0:1789760796.859136:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.859151:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18548, 18548] 00020000:00200000:1.0:1789760796.859159:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75972608 chunk: [75972608, 75976704) 75976704, 4096 00020000:00200000:1.0:1789760796.859163:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.859179:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18548, 18548] 00000008:00010000:1.0:1789760796.859193:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.859198:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.859206:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75972608, 75976704) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.859218:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75972608=0x4874000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.859264:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.859267:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75976704 00000020:00010000:1.0:1789760796.859273:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.859276:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.859279:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.859283:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.859286:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.859292:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.859298:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.859312:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760796.859630:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 75972608<>75976704 00000100:00100000:0.0:1789760796.859767:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565605376:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.875590:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985856 gen 3 00000008:00010000:0.0:1789760796.875630:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760796.875636:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.875671:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.875688:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760796.875710:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565605376:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.875809:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18548, 18548] 00000080:00200000:1.0:1789760796.875822:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.875827:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.875838:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75976704 i_blocks 148416 00000020:00200000:1.0:1789760796.875842:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.875850:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.875876:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75976704, count: 4096 00000080:00200000:1.0:1789760796.875882:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75976704, bytes: 4096 00000080:00200000:1.0:1789760796.875901:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.875909:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.875911:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75976704, 75980800) 0 0 00000080:00200000:1.0:1789760796.875915:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.875929:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18549, 18549] 00020000:00200000:1.0:1789760796.875935:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75976704 chunk: [75976704, 75980800) 75980800, 4096 00020000:00200000:1.0:1789760796.875939:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.875953:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18549, 18549] 00000008:00010000:1.0:1789760796.875964:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.875969:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.875977:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75976704, 75980800) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.875987:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75976704=0x4875000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.876025:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.876027:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75980800 00000020:00010000:1.0:1789760796.876034:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.876037:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.876040:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.876043:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.876045:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.876051:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.876056:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760796.876393:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:0.0:1789760796.876492:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 75976704<>75980800 00000100:00100000:0.0:1789760796.876540:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565605504:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.896159:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985857 gen 3 00000008:00010000:0.0:1789760796.896195:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760796.896201:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.896210:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.896228:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760796.896252:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565605504:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760796.896397:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18549, 18549] 00000080:00200000:1.0:1789760796.896410:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760796.896415:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760796.896426:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75980800 i_blocks 148424 00000020:00200000:1.0:1789760796.896430:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760796.896437:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760796.896460:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75980800, count: 4096 00000080:00200000:1.0:1789760796.896466:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75980800, bytes: 4096 00000080:00200000:1.0:1789760796.896473:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760796.896480:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760796.896482:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75980800, 75984896) 0 0 00000080:00200000:1.0:1789760796.896486:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760796.896499:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18550, 18550] 00020000:00200000:1.0:1789760796.896506:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75980800 chunk: [75980800, 75984896) 75984896, 4096 00020000:00200000:1.0:1789760796.896510:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760796.896523:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18550, 18550] 00000008:00010000:1.0:1789760796.896535:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760796.896539:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760796.896547:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75980800, 75984896) append 0 append lockpos 0 00000080:00200000:1.0:1789760796.896558:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75980800=0x4876000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760796.896600:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760796.896603:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75984896 00000020:00010000:1.0:1789760796.896609:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760796.896612:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.896615:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760796.896618:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760796.896620:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760796.896626:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760796.896631:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760796.896678:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760796.898002:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 75980800<>75984896 00000100:00100000:1.0:1789760796.898062:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565605632:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760796.913523:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985858 gen 3 00000008:00010000:0.0:1789760796.913562:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760796.913569:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760796.913578:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760796.913600:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760796.913623:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565605632:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.913792:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18550, 18550] 00000080:00200000:2.0:1789760796.913804:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.913809:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.913821:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75984896 i_blocks 148432 00000020:00200000:2.0:1789760796.913824:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.913833:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.913860:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75984896, count: 4096 00000080:00200000:2.0:1789760796.913866:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75984896, bytes: 4096 00000080:00200000:2.0:1789760796.913873:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.913880:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.913882:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75984896, 75988992) 0 0 00000080:00200000:2.0:1789760796.913886:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.913901:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18551, 18551] 00020000:00200000:2.0:1789760796.913908:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75984896 chunk: [75984896, 75988992) 75988992, 4096 00020000:00200000:2.0:1789760796.913912:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.913926:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18551, 18551] 00000008:00010000:2.0:1789760796.913941:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.913946:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.913955:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75984896, 75988992) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.913976:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75984896=0x4877000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.914020:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.914023:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75988992 00000020:00010000:2.0:1789760796.914029:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.914032:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.914034:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.914037:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.914040:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.914046:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.914051:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.914064:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f480 00000008:00100000:2.0:1789760796.914156:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 75984896<>75988992 00000100:00100000:3.0:1789760796.914289:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565605760:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.925470:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985859 gen 3 00000008:00010000:3.0:1789760796.925512:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f480 00000020:00200000:3.0:1789760796.925518:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.925526:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.925592:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f480 00000100:00100000:3.0:1789760796.925615:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565605760:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.931444:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18551, 18551] 00000080:00200000:2.0:1789760796.931466:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.931471:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.931484:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75988992 i_blocks 148440 00000020:00200000:2.0:1789760796.931712:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.931728:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.931754:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75988992, count: 4096 00000080:00200000:2.0:1789760796.931762:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75988992, bytes: 4096 00000080:00200000:2.0:1789760796.931811:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.931820:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.931822:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75988992, 75993088) 0 0 00000080:00200000:2.0:1789760796.931827:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.931840:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18552, 18552] 00020000:00200000:2.0:1789760796.931846:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75988992 chunk: [75988992, 75993088) 75993088, 4096 00020000:00200000:2.0:1789760796.931850:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.931866:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18552, 18552] 00000008:00010000:2.0:1789760796.931880:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.931884:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.931894:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75988992, 75993088) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.931904:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75988992=0x4878000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.932095:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.932099:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75993088 00000020:00010000:2.0:1789760796.932106:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.932109:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.932112:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.932116:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.932118:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.932125:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.932130:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.932155:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760796.932268:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 75988992<>75993088 00000100:00100000:3.0:1789760796.932458:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565605888:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.947719:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985860 gen 3 00000008:00010000:3.0:1789760796.947755:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760796.947762:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.947770:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.947785:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760796.947808:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565605888:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.948917:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18552, 18552] 00000080:00200000:2.0:1789760796.948932:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.948938:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.948950:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75993088 i_blocks 148448 00000020:00200000:2.0:1789760796.948954:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.948965:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.949000:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75993088, count: 4096 00000080:00200000:2.0:1789760796.949006:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75993088, bytes: 4096 00000080:00200000:2.0:1789760796.949015:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.949023:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.949025:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75993088, 75997184) 0 0 00000080:00200000:2.0:1789760796.949028:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.949041:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18553, 18553] 00020000:00200000:2.0:1789760796.949048:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75993088 chunk: [75993088, 75997184) 75997184, 4096 00020000:00200000:2.0:1789760796.949069:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.949084:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18553, 18553] 00000008:00010000:2.0:1789760796.949103:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.949109:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.949117:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75993088, 75997184) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.949127:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75993088=0x4879000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.949169:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.949171:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 75997184 00000020:00010000:2.0:1789760796.949177:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.949180:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.949183:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.949186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.949188:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.949194:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.949199:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.949213:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760796.949305:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 75993088<>75997184 00000100:00100000:3.0:1789760796.950903:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565606016:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.965150:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985861 gen 3 00000008:00010000:3.0:1789760796.965188:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760796.965194:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.965202:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.965225:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760796.965251:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565606016:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.965259:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18553, 18553] 00000080:00200000:2.0:1789760796.965273:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.965278:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.965290:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 75997184 i_blocks 148456 00000020:00200000:2.0:1789760796.965294:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.965302:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.965327:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 75997184, count: 4096 00000080:00200000:2.0:1789760796.965333:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 75997184, bytes: 4096 00000080:00200000:2.0:1789760796.965341:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.965348:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.965350:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [75997184, 76001280) 0 0 00000080:00200000:2.0:1789760796.965354:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.965367:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18554, 18554] 00020000:00200000:2.0:1789760796.965383:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 75997184 chunk: [75997184, 76001280) 76001280, 4096 00020000:00200000:2.0:1789760796.965387:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.965404:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18554, 18554] 00000008:00010000:2.0:1789760796.965417:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.965422:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.965431:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [75997184, 76001280) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.965442:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=75997184=0x487a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.965487:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.965489:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76001280 00000020:00010000:2.0:1789760796.965496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.965499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.965501:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.965504:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.965506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.965512:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.965518:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.965530:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760796.965623:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 75997184<>76001280 00000100:00100000:3.0:1789760796.965688:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565606144:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760796.985969:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985862 gen 3 00000008:00010000:3.0:1789760796.986121:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:3.0:1789760796.986128:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760796.986136:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760796.986158:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:3.0:1789760796.986188:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565606144:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760796.986287:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18554, 18554] 00000080:00200000:2.0:1789760796.986299:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760796.986304:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760796.986317:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76001280 i_blocks 148464 00000020:00200000:2.0:1789760796.986321:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760796.986330:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760796.986355:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76001280, count: 4096 00000080:00200000:2.0:1789760796.986360:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76001280, bytes: 4096 00000080:00200000:2.0:1789760796.986368:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760796.986384:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760796.986386:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76001280, 76005376) 0 0 00000080:00200000:2.0:1789760796.986389:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760796.986404:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18555, 18555] 00020000:00200000:2.0:1789760796.986411:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76001280 chunk: [76001280, 76005376) 76005376, 4096 00020000:00200000:2.0:1789760796.986415:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760796.986429:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18555, 18555] 00000008:00010000:2.0:1789760796.986442:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760796.986447:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760796.986455:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76001280, 76005376) append 0 append lockpos 0 00000080:00200000:2.0:1789760796.986466:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76001280=0x487b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760796.986508:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760796.986511:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76005376 00000020:00010000:2.0:1789760796.986517:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760796.986521:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.986524:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760796.986528:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760796.986531:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760796.986537:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760796.986542:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760796.986556:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760796.986882:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 76001280<>76005376 00000100:00100000:2.0:1789760796.986920:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565606272:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.006114:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985863 gen 3 00000008:00010000:2.0:1789760797.006149:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760797.006156:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.006164:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.006184:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760797.006208:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565606272:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.006404:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18555, 18555] 00000080:00200000:3.0:1789760797.006417:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.006422:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.006433:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76005376 i_blocks 148472 00000020:00200000:3.0:1789760797.006436:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.006443:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.006464:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76005376, count: 4096 00000080:00200000:3.0:1789760797.006469:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76005376, bytes: 4096 00000080:00200000:3.0:1789760797.006476:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.006483:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.006485:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76005376, 76009472) 0 0 00000080:00200000:3.0:1789760797.006489:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.006502:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18556, 18556] 00020000:00200000:3.0:1789760797.006510:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76005376 chunk: [76005376, 76009472) 76009472, 4096 00020000:00200000:3.0:1789760797.006514:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760797.006529:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18556, 18556] 00000008:00010000:3.0:1789760797.006542:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760797.006547:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760797.006557:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76005376, 76009472) append 0 append lockpos 0 00000080:00200000:3.0:1789760797.006569:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76005376=0x487c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760797.006608:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760797.006610:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76009472 00000020:00010000:3.0:1789760797.006616:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:3.0:1789760797.006619:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.006622:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.006625:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760797.006628:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760797.006634:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760797.006639:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760797.006687:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362180 00000008:00100000:3.0:1789760797.006783:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65500 - object 0x240000400:5 offset 76005376<>76009472 00000100:00100000:2.0:1789760797.006904:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760797.006917:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565606400:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.021846:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985864 gen 3 00000008:00010000:2.0:1789760797.021882:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362180 00000020:00200000:2.0:1789760797.021889:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.021898:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.021961:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362180 00000100:00100000:2.0:1789760797.021985:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565606400:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.025059:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18556, 18556] 00000080:00200000:3.0:1789760797.025079:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.025084:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.025246:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76009472 i_blocks 148480 00000020:00200000:2.0:1789760797.025253:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.025267:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.025292:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76009472, count: 4096 00000080:00200000:2.0:1789760797.025299:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76009472, bytes: 4096 00000080:00200000:2.0:1789760797.025320:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.025327:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.025329:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76009472, 76013568) 0 0 00000080:00200000:2.0:1789760797.025332:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.025345:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18557, 18557] 00020000:00200000:2.0:1789760797.025352:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76009472 chunk: [76009472, 76013568) 76013568, 4096 00020000:00200000:2.0:1789760797.025356:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.025381:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18557, 18557] 00000008:00010000:2.0:1789760797.025394:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.025400:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.025409:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76009472, 76013568) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.025420:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76009472=0x487d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.025469:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.025473:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76013568 00000020:00010000:2.0:1789760797.025479:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.025482:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.025485:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.025488:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420ec68 (ffff93f84354e878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.025490:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.025497:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.025842:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.025889:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760797.025989:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 76009472<>76013568 00000100:00100000:2.0:1789760797.026035:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760797.026052:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565606528:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.047106:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985865 gen 3 00000008:00010000:2.0:1789760797.047145:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:2.0:1789760797.047152:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.047160:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.047180:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:2.0:1789760797.047203:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565606528:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.047293:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18557, 18557] 00000080:00200000:3.0:1789760797.047310:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.047315:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.047327:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76013568 i_blocks 148488 00000020:00200000:3.0:1789760797.047330:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.047338:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.047397:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76013568, count: 4096 00000080:00200000:3.0:1789760797.047405:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76013568, bytes: 4096 00000080:00200000:3.0:1789760797.047413:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.047420:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.047422:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76013568, 76017664) 0 0 00000080:00200000:3.0:1789760797.047426:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.047441:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18558, 18558] 00020000:00200000:3.0:1789760797.047449:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76013568 chunk: [76013568, 76017664) 76017664, 4096 00020000:00200000:3.0:1789760797.047453:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760797.047468:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18558, 18558] 00000008:00010000:3.0:1789760797.047482:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760797.047487:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760797.047494:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76013568, 76017664) append 0 append lockpos 0 00000080:00200000:3.0:1789760797.047505:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76013568=0x487e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760797.047549:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760797.047552:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76017664 00000020:00010000:3.0:1789760797.047559:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:3.0:1789760797.047563:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.047566:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.047569:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760797.047572:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760797.047578:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760797.047583:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760797.047597:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760797.047731:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 76013568<>76017664 00000100:00100000:3.0:1789760797.047777:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760797.047791:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565606656:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.064625:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985866 gen 3 00000008:00010000:3.0:1789760797.064700:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:3.0:1789760797.064707:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.064714:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.064726:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:3.0:1789760797.064747:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565606656:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760797.064840:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18558, 18558] 00000080:00200000:0.0:1789760797.064852:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760797.064857:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760797.064868:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76017664 i_blocks 148496 00000020:00200000:0.0:1789760797.064872:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760797.064881:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760797.064905:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76017664, count: 4096 00000080:00200000:0.0:1789760797.064912:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76017664, bytes: 4096 00000080:00200000:0.0:1789760797.064920:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760797.064927:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760797.064930:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76017664, 76021760) 0 0 00000080:00200000:0.0:1789760797.064933:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760797.064947:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18559, 18559] 00020000:00200000:0.0:1789760797.064953:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76017664 chunk: [76017664, 76021760) 76021760, 4096 00020000:00200000:0.0:1789760797.064957:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760797.064973:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18559, 18559] 00000008:00010000:0.0:1789760797.064987:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760797.064992:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760797.065000:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76017664, 76021760) append 0 append lockpos 0 00000080:00200000:0.0:1789760797.065011:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76017664=0x487f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760797.065054:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760797.065057:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76021760 00000020:00010000:0.0:1789760797.065063:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:0.0:1789760797.065066:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.065069:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.065072:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760797.065075:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760797.065081:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760797.065086:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760797.065111:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760797.065207:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76017664<>76021760 00000100:00100000:0.0:1789760797.065250:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760797.065266:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565606784:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.078764:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985867 gen 3 00000008:00010000:0.0:1789760797.078798:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760797.078804:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.078812:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.078833:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760797.078855:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565606784:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.078954:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18559, 18559] 00000080:00200000:3.0:1789760797.078966:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.078971:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.078983:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76021760 i_blocks 148504 00000020:00200000:3.0:1789760797.078986:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.078997:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.079023:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76021760, count: 4096 00000080:00200000:3.0:1789760797.079030:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76021760, bytes: 4096 00000080:00200000:3.0:1789760797.079038:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.079045:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.079047:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76021760, 76025856) 0 0 00000080:00200000:3.0:1789760797.079051:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.079062:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18560, 18560] 00020000:00200000:3.0:1789760797.079069:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76021760 chunk: [76021760, 76025856) 76025856, 4096 00020000:00200000:3.0:1789760797.079073:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760797.079200:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18560, 18560] 00000008:00010000:3.0:1789760797.079215:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760797.079220:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760797.079229:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76021760, 76025856) append 0 append lockpos 0 00000080:00200000:3.0:1789760797.079239:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76021760=0x4880000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760797.079283:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760797.079285:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76025856 00000020:00010000:3.0:1789760797.079291:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:3.0:1789760797.079294:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.079297:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.079299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760797.079302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760797.080443:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760797.080453:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760797.080476:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362a80 00000008:00100000:3.0:1789760797.080577:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64380 - object 0x240000400:5 offset 76021760<>76025856 00000100:00100000:2.0:1789760797.081217:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565606912:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.098303:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985868 gen 3 00000008:00010000:2.0:1789760797.098340:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362a80 00000020:00200000:2.0:1789760797.098346:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.098354:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.098386:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362a80 00000100:00100000:2.0:1789760797.098410:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565606912:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.099059:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18560, 18560] 00000080:00200000:3.0:1789760797.099072:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.099078:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.099088:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76025856 i_blocks 148512 00000020:00200000:3.0:1789760797.099092:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.099109:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.099135:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76025856, count: 4096 00000080:00200000:3.0:1789760797.099141:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76025856, bytes: 4096 00000080:00200000:3.0:1789760797.099151:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.099158:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.099161:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76025856, 76029952) 0 0 00000080:00200000:3.0:1789760797.099165:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.099177:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18561, 18561] 00020000:00200000:3.0:1789760797.099186:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76025856 chunk: [76025856, 76029952) 76029952, 4096 00020000:00200000:3.0:1789760797.099189:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760797.099205:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18561, 18561] 00000008:00010000:3.0:1789760797.099219:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760797.099224:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760797.099233:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76025856, 76029952) append 0 append lockpos 0 00000080:00200000:3.0:1789760797.099243:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76025856=0x4881000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760797.099290:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760797.099292:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76029952 00000020:00010000:3.0:1789760797.099299:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:3.0:1789760797.099302:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.099304:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.099307:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760797.099310:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760797.099316:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760797.099321:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760797.099336:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a3626c0 00000008:00100000:3.0:1789760797.099457:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65f80 - object 0x240000400:5 offset 76025856<>76029952 00000100:00100000:2.0:1789760797.099718:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760797.099736:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565607040:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.127243:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985869 gen 3 00000008:00010000:2.0:1789760797.127281:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a3626c0 00000020:00200000:2.0:1789760797.127288:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.127297:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.127316:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a3626c0 00000100:00100000:2.0:1789760797.127338:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565607040:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.127402:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18561, 18561] 00000080:00200000:3.0:1789760797.127415:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.127429:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.127441:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76029952 i_blocks 148520 00000020:00200000:3.0:1789760797.127498:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.127515:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.127540:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76029952, count: 4096 00000080:00200000:3.0:1789760797.127546:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76029952, bytes: 4096 00000080:00200000:3.0:1789760797.127554:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.127560:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.127562:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76029952, 76034048) 0 0 00000080:00200000:3.0:1789760797.127565:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.127578:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18562, 18562] 00020000:00200000:3.0:1789760797.127585:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76029952 chunk: [76029952, 76034048) 76034048, 4096 00020000:00200000:3.0:1789760797.127589:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760797.129923:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18562, 18562] 00000008:00010000:0.0:1789760797.129954:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760797.129960:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760797.129972:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76029952, 76034048) append 0 append lockpos 0 00000080:00200000:0.0:1789760797.129984:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76029952=0x4882000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760797.130044:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760797.130047:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76034048 00000020:00010000:0.0:1789760797.130054:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:0.0:1789760797.130071:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.130080:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.130084:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760797.130087:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760797.130095:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760797.130102:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760797.130120:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760797.130221:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76029952<>76034048 00000100:00100000:1.0:1789760797.130513:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760797.130538:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565607168:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.141303:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985870 gen 3 00000008:00010000:0.0:1789760797.141348:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760797.141355:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.141364:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.141391:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760797.141415:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565607168:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.141590:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18562, 18562] 00000080:00200000:1.0:1789760797.141605:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.141609:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.141623:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76034048 i_blocks 148528 00000020:00200000:1.0:1789760797.141626:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.141635:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.141691:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76034048, count: 4096 00000080:00200000:1.0:1789760797.141698:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76034048, bytes: 4096 00000080:00200000:1.0:1789760797.141706:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.141714:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.141716:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76034048, 76038144) 0 0 00000080:00200000:1.0:1789760797.141729:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.141742:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18563, 18563] 00020000:00200000:1.0:1789760797.141750:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76034048 chunk: [76034048, 76038144) 76038144, 4096 00020000:00200000:1.0:1789760797.141754:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.141769:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18563, 18563] 00000008:00010000:1.0:1789760797.141782:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.141787:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.141795:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76034048, 76038144) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.141806:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76034048=0x4883000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.141849:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.141852:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76038144 00000020:00010000:1.0:1789760797.141859:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.141862:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.141865:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.141868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.141870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.141877:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.141883:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.141897:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760797.141999:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 76034048<>76038144 00000100:00100000:1.0:1789760797.142044:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760797.142060:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565607296:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.147474:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565607552:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760797.147583:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1789760797.147591:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:af634c28-3647-4f73-bb24-40d7878858fb:2359:1876699565607424:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760797.147617:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2359:1876699565607680:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760797.153462:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2359:1876699565607680:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760797.154583:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565607552:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760797.155498:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:af634c28-3647-4f73-bb24-40d7878858fb:2359:1876699565607424:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1789760797.163753:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985871 gen 3 00000008:00010000:0.0:1789760797.163788:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760797.163795:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.163804:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.163825:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760797.163848:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565607296:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.163945:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18563, 18563] 00000080:00200000:1.0:1789760797.163958:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.163963:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.163974:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76038144 i_blocks 148536 00000020:00200000:1.0:1789760797.163977:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.163986:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.164014:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76038144, count: 4096 00000080:00200000:1.0:1789760797.164020:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76038144, bytes: 4096 00000080:00200000:1.0:1789760797.164029:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.164037:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.164039:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76038144, 76042240) 0 0 00000080:00200000:1.0:1789760797.164043:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.164056:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18564, 18564] 00020000:00200000:1.0:1789760797.164063:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76038144 chunk: [76038144, 76042240) 76042240, 4096 00020000:00200000:1.0:1789760797.164067:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.164082:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18564, 18564] 00000008:00010000:1.0:1789760797.164097:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.164101:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.164110:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76038144, 76042240) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.164121:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76038144=0x4884000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.164166:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.164169:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76042240 00000020:00010000:1.0:1789760797.164175:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.164178:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.164181:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.164184:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.164186:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.164192:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.164198:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.164213:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760797.164324:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 76038144<>76042240 00000100:00100000:0.0:1789760797.164479:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760797.164504:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565607808:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.183856:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985872 gen 3 00000008:00010000:0.0:1789760797.183892:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760797.183898:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.183907:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.183924:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760797.183947:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565607808:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.184391:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18564, 18564] 00000080:00200000:1.0:1789760797.184404:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.184409:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.184420:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76042240 i_blocks 148544 00000020:00200000:1.0:1789760797.184423:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.184431:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.184456:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76042240, count: 4096 00000080:00200000:1.0:1789760797.184462:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76042240, bytes: 4096 00000080:00200000:1.0:1789760797.184469:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.184476:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.184479:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76042240, 76046336) 0 0 00000080:00200000:1.0:1789760797.184483:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.184496:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18565, 18565] 00020000:00200000:1.0:1789760797.184503:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76042240 chunk: [76042240, 76046336) 76046336, 4096 00020000:00200000:1.0:1789760797.184508:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.184522:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18565, 18565] 00000008:00010000:1.0:1789760797.184535:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.184539:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.184549:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76042240, 76046336) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.184560:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76042240=0x4885000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.184614:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.184617:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76046336 00000020:00010000:1.0:1789760797.184624:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.184626:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.184629:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.184631:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.184730:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.184739:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.184745:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.184758:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760797.184856:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 76042240<>76046336 00000100:00100000:0.0:1789760797.184982:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565607936:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760797.210928:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760797.214735:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985873 gen 3 00000008:00010000:0.0:1789760797.214776:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760797.214784:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.214793:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.214807:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760797.214830:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565607936:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.216473:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18565, 18565] 00000080:00200000:1.0:1789760797.216491:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.216497:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.216523:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76046336 i_blocks 148552 00000020:00200000:1.0:1789760797.216528:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.216538:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.216562:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76046336, count: 4096 00000080:00200000:1.0:1789760797.216568:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76046336, bytes: 4096 00000080:00200000:1.0:1789760797.216579:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.216586:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.216588:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76046336, 76050432) 0 0 00000080:00200000:1.0:1789760797.216592:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.216605:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18566, 18566] 00020000:00200000:1.0:1789760797.216612:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76046336 chunk: [76046336, 76050432) 76050432, 4096 00020000:00200000:1.0:1789760797.216616:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.216631:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18566, 18566] 00000008:00010000:1.0:1789760797.216671:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.216676:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.216685:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76046336, 76050432) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.216695:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76046336=0x4886000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.216744:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.216747:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76050432 00000020:00010000:1.0:1789760797.216753:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.216756:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.216759:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.216762:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.216764:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.216771:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.216776:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.216793:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760797.216891:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 76046336<>76050432 00000100:00100000:0.0:1789760797.217094:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760797.217114:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565608064:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.240894:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985874 gen 3 00000008:00010000:0.0:1789760797.240928:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760797.240935:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.240943:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.240962:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760797.240985:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565608064:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.241078:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18566, 18566] 00000080:00200000:1.0:1789760797.241089:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.241094:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.241106:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76050432 i_blocks 148560 00000020:00200000:1.0:1789760797.241109:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.241119:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.241146:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76050432, count: 4096 00000080:00200000:1.0:1789760797.241151:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76050432, bytes: 4096 00000080:00200000:1.0:1789760797.241158:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.241165:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.241167:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76050432, 76054528) 0 0 00000080:00200000:1.0:1789760797.241170:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.241183:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18567, 18567] 00020000:00200000:1.0:1789760797.241190:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76050432 chunk: [76050432, 76054528) 76054528, 4096 00020000:00200000:1.0:1789760797.241194:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.241208:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18567, 18567] 00000008:00010000:1.0:1789760797.241221:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.241226:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.241233:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76050432, 76054528) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.241242:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76050432=0x4887000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.241283:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.241286:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76054528 00000020:00010000:1.0:1789760797.241292:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.241295:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.241297:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.241301:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.241498:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.241556:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.241564:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.241578:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760797.241818:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 76050432<>76054528 00000100:00100000:0.0:1789760797.241870:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565608192:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.268731:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985875 gen 3 00000008:00010000:0.0:1789760797.268769:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760797.268776:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.268784:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.268810:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760797.268834:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565608192:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.268972:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18567, 18567] 00000080:00200000:2.0:1789760797.268985:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.268990:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.269002:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76054528 i_blocks 148568 00000020:00200000:2.0:1789760797.269005:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.269014:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.269040:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76054528, count: 4096 00000080:00200000:2.0:1789760797.269046:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76054528, bytes: 4096 00000080:00200000:2.0:1789760797.269054:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.269062:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.269064:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76054528, 76058624) 0 0 00000080:00200000:2.0:1789760797.269068:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.269082:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18568, 18568] 00020000:00200000:2.0:1789760797.269089:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76054528 chunk: [76054528, 76058624) 76058624, 4096 00020000:00200000:2.0:1789760797.269093:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.269108:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18568, 18568] 00000008:00010000:2.0:1789760797.269121:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.269126:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.269135:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76054528, 76058624) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.269145:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76054528=0x4888000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.269190:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.269193:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76058624 00000020:00010000:2.0:1789760797.269200:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.269204:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.269206:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.269209:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.269212:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.269218:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.269223:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.269236:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760797.269332:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 76054528<>76058624 00000100:00100000:3.0:1789760797.271237:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760797.271266:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565608320:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.291108:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985876 gen 3 00000008:00010000:3.0:1789760797.291155:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760797.291163:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.291173:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.291195:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760797.291220:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565608320:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.291337:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18568, 18568] 00000080:00200000:2.0:1789760797.291352:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.291357:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.291379:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76058624 i_blocks 148576 00000020:00200000:2.0:1789760797.291382:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.291392:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.291421:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76058624, count: 4096 00000080:00200000:2.0:1789760797.291428:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76058624, bytes: 4096 00000080:00200000:2.0:1789760797.291436:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.291443:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.291445:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76058624, 76062720) 0 0 00000080:00200000:2.0:1789760797.291450:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.291464:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18569, 18569] 00020000:00200000:2.0:1789760797.291471:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76058624 chunk: [76058624, 76062720) 76062720, 4096 00020000:00200000:2.0:1789760797.291476:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.291491:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18569, 18569] 00000008:00010000:2.0:1789760797.291505:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.291511:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.291520:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76058624, 76062720) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.291531:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76058624=0x4889000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.291574:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.291577:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76062720 00000020:00010000:2.0:1789760797.291583:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.291586:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.291589:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.291592:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.291594:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.291601:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.291606:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.291620:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354ff00 00000008:00100000:2.0:1789760797.291871:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 76058624<>76062720 00000100:00100000:2.0:1789760797.291914:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565608448:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.320946:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985877 gen 3 00000008:00010000:2.0:1789760797.320986:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354ff00 00000020:00200000:2.0:1789760797.320993:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.321002:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.321070:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354ff00 00000100:00100000:2.0:1789760797.321095:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565608448:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.323765:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18569, 18569] 00000080:00200000:3.0:1789760797.323788:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.323793:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.323809:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76062720 i_blocks 148584 00000020:00200000:3.0:1789760797.323813:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.323829:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.324173:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76062720, count: 4096 00000080:00200000:3.0:1789760797.324182:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76062720, bytes: 4096 00000080:00200000:3.0:1789760797.324196:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.324203:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.324205:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76062720, 76066816) 0 0 00000080:00200000:3.0:1789760797.324209:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.324222:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18570, 18570] 00020000:00200000:3.0:1789760797.324229:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76062720 chunk: [76062720, 76066816) 76066816, 4096 00020000:00200000:3.0:1789760797.324233:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760797.324249:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18570, 18570] 00000008:00010000:3.0:1789760797.324263:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760797.324268:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760797.324277:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76062720, 76066816) append 0 append lockpos 0 00000080:00200000:3.0:1789760797.324287:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76062720=0x488a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760797.324340:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760797.324343:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76066816 00000020:00010000:3.0:1789760797.324350:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:3.0:1789760797.324353:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.324356:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.324359:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760797.324361:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760797.324383:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760797.324389:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760797.324411:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362240 00000008:00100000:3.0:1789760797.324529:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66d80 - object 0x240000400:5 offset 76062720<>76066816 00000100:00100000:2.0:1789760797.324614:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565608576:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.339297:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985878 gen 3 00000008:00010000:2.0:1789760797.339345:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362240 00000020:00200000:2.0:1789760797.339351:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.339361:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.339394:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362240 00000100:00100000:2.0:1789760797.339426:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565608576:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.340201:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18570, 18570] 00000080:00200000:3.0:1789760797.340217:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.340222:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.340235:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76066816 i_blocks 148592 00000020:00200000:3.0:1789760797.340238:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.340247:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.340275:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76066816, count: 4096 00000080:00200000:3.0:1789760797.340281:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76066816, bytes: 4096 00000080:00200000:3.0:1789760797.340290:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.340298:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.340301:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76066816, 76070912) 0 0 00000080:00200000:3.0:1789760797.340314:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.340329:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18571, 18571] 00020000:00200000:3.0:1789760797.340337:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76066816 chunk: [76066816, 76070912) 76070912, 4096 00020000:00200000:3.0:1789760797.340341:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760797.340358:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18571, 18571] 00000008:00010000:3.0:1789760797.340385:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760797.340390:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760797.340400:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76066816, 76070912) append 0 append lockpos 0 00000080:00200000:3.0:1789760797.340562:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76066816=0x488b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760797.340617:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760797.340620:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76070912 00000020:00010000:3.0:1789760797.340627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:3.0:1789760797.340630:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.340633:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.340636:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760797.340639:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760797.341180:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760797.341188:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760797.341211:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362540 00000008:00100000:3.0:1789760797.341319:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d64a80 - object 0x240000400:5 offset 76066816<>76070912 00000100:00100000:2.0:1789760797.341527:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565608704:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.356077:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985879 gen 3 00000008:00010000:2.0:1789760797.356116:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362540 00000020:00200000:2.0:1789760797.356122:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.356131:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.356191:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362540 00000100:00100000:2.0:1789760797.356214:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565608704:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.356380:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18571, 18571] 00000080:00200000:3.0:1789760797.356394:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.356400:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.356468:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76070912 i_blocks 148600 00000020:00200000:3.0:1789760797.356472:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.356480:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.356574:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76070912, count: 4096 00000080:00200000:3.0:1789760797.356581:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76070912, bytes: 4096 00000080:00200000:3.0:1789760797.356589:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.356595:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.356597:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76070912, 76075008) 0 0 00000080:00200000:3.0:1789760797.356601:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.356614:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18572, 18572] 00020000:00200000:3.0:1789760797.356621:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76070912 chunk: [76070912, 76075008) 76075008, 4096 00020000:00200000:3.0:1789760797.356624:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760797.356638:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18572, 18572] 00000008:00010000:2.0:1789760797.357051:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.357057:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.357066:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76070912, 76075008) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.357078:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76070912=0x488c000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.357120:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.357122:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76075008 00000020:00010000:2.0:1789760797.357130:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.357133:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.357136:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.357139:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.357142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.357149:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.357155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.357167:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354ff00 00000008:00100000:2.0:1789760797.357580:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 76070912<>76075008 00000100:00100000:3.0:1789760797.357665:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565608832:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.370932:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985880 gen 3 00000008:00010000:3.0:1789760797.370967:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354ff00 00000020:00200000:3.0:1789760797.370973:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.370982:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.371056:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354ff00 00000100:00100000:3.0:1789760797.371081:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565608832:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.371208:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18572, 18572] 00000080:00200000:2.0:1789760797.371222:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.371227:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.371239:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76075008 i_blocks 148608 00000020:00200000:2.0:1789760797.371242:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.371250:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.371285:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76075008, count: 4096 00000080:00200000:2.0:1789760797.371292:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76075008, bytes: 4096 00000080:00200000:2.0:1789760797.371299:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.371306:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.371309:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76075008, 76079104) 0 0 00000080:00200000:2.0:1789760797.371312:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.371326:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18573, 18573] 00020000:00200000:2.0:1789760797.371333:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76075008 chunk: [76075008, 76079104) 76079104, 4096 00020000:00200000:2.0:1789760797.371336:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.371352:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18573, 18573] 00000008:00010000:2.0:1789760797.371364:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.371378:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.371387:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76075008, 76079104) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.371397:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76075008=0x488d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.371604:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.371607:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76079104 00000020:00010000:2.0:1789760797.371614:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.371618:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.371621:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.371624:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.371627:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.371632:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.371854:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.371873:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fa80 00000008:00100000:2.0:1789760797.371971:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 76075008<>76079104 00000100:00100000:3.0:1789760797.372042:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565608960:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.394352:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985881 gen 3 00000008:00010000:3.0:1789760797.394397:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fa80 00000020:00200000:3.0:1789760797.394403:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.394411:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.394430:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fa80 00000100:00100000:3.0:1789760797.394455:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565608960:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.395527:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18573, 18573] 00000080:00200000:2.0:1789760797.395544:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.395549:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.395563:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76079104 i_blocks 148616 00000020:00200000:2.0:1789760797.395567:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.395577:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.395603:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76079104, count: 4096 00000080:00200000:2.0:1789760797.395610:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76079104, bytes: 4096 00000080:00200000:2.0:1789760797.395621:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.395630:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.395632:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76079104, 76083200) 0 0 00000080:00200000:2.0:1789760797.395636:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.395688:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18574, 18574] 00020000:00200000:2.0:1789760797.395695:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76079104 chunk: [76079104, 76083200) 76083200, 4096 00020000:00200000:2.0:1789760797.395698:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.395714:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18574, 18574] 00000008:00010000:2.0:1789760797.395727:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.395732:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.395741:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76079104, 76083200) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.395752:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76079104=0x488e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.396204:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.396208:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76083200 00000020:00010000:2.0:1789760797.396216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.396220:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.396223:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.396227:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.396230:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.396239:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.396245:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.396335:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760797.396445:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0e00 - object 0x240000400:5 offset 76079104<>76083200 00000100:00100000:3.0:1789760797.397246:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760797.397341:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565609088:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.418181:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985882 gen 3 00000008:00010000:3.0:1789760797.418220:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760797.418227:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.418236:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.418255:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760797.418279:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565609088:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.418381:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18574, 18574] 00000080:00200000:2.0:1789760797.418394:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.418399:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.418410:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76083200 i_blocks 148624 00000020:00200000:2.0:1789760797.418413:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.418421:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.418447:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76083200, count: 4096 00000080:00200000:2.0:1789760797.418453:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76083200, bytes: 4096 00000080:00200000:2.0:1789760797.418460:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.418468:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.418470:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76083200, 76087296) 0 0 00000080:00200000:2.0:1789760797.418473:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.418486:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18575, 18575] 00020000:00200000:2.0:1789760797.418492:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76083200 chunk: [76083200, 76087296) 76087296, 4096 00020000:00200000:2.0:1789760797.418496:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.418511:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18575, 18575] 00000008:00010000:2.0:1789760797.418523:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.418528:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.418538:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76083200, 76087296) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.418548:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76083200=0x488f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.418591:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.418594:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76087296 00000020:00010000:2.0:1789760797.418601:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.418604:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.418607:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.418610:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.418612:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.418618:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.418623:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.418637:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760797.418774:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3800 - object 0x240000400:5 offset 76083200<>76087296 00000100:00100000:3.0:1789760797.418923:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565609216:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.435376:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985883 gen 3 00000008:00010000:3.0:1789760797.435418:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:3.0:1789760797.435425:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.435434:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.435453:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:3.0:1789760797.435475:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565609216:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.435617:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18575, 18575] 00000080:00200000:2.0:1789760797.435630:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.435635:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.435680:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76087296 i_blocks 148632 00000020:00200000:2.0:1789760797.435684:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.435692:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.435718:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76087296, count: 4096 00000080:00200000:2.0:1789760797.435725:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76087296, bytes: 4096 00000080:00200000:2.0:1789760797.435732:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.435739:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.435741:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76087296, 76091392) 0 0 00000080:00200000:2.0:1789760797.435744:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.435756:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18576, 18576] 00020000:00200000:2.0:1789760797.435763:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76087296 chunk: [76087296, 76091392) 76091392, 4096 00020000:00200000:2.0:1789760797.435767:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.435782:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18576, 18576] 00000008:00010000:2.0:1789760797.435795:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.435799:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.435809:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76087296, 76091392) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.435819:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76087296=0x4890000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.435860:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.435863:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76091392 00000020:00010000:2.0:1789760797.435870:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.435873:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.435876:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.435879:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.435881:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.435887:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.435893:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.435905:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f300 00000008:00100000:2.0:1789760797.435996:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c3480 - object 0x240000400:5 offset 76087296<>76091392 00000100:00100000:3.0:1789760797.436117:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760797.436130:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565609344:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.450820:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985884 gen 3 00000008:00010000:3.0:1789760797.450853:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f300 00000020:00200000:3.0:1789760797.450859:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.450866:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.450886:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f300 00000100:00100000:3.0:1789760797.450907:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565609344:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.450995:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18576, 18576] 00000080:00200000:2.0:1789760797.451008:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.451012:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.451022:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76091392 i_blocks 148640 00000020:00200000:2.0:1789760797.451025:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.451033:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.451058:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76091392, count: 4096 00000080:00200000:2.0:1789760797.451063:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76091392, bytes: 4096 00000080:00200000:2.0:1789760797.451070:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.451077:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.451079:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76091392, 76095488) 0 0 00000080:00200000:2.0:1789760797.451082:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.451094:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18577, 18577] 00020000:00200000:2.0:1789760797.451100:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76091392 chunk: [76091392, 76095488) 76095488, 4096 00020000:00200000:2.0:1789760797.451104:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.451117:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18577, 18577] 00000008:00010000:2.0:1789760797.451129:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.451132:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.451140:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76091392, 76095488) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.451389:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76091392=0x4891000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.451447:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.451450:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76095488 00000020:00010000:2.0:1789760797.451456:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.451459:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.451461:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.451464:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.451466:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.451473:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.451479:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.451493:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760797.451588:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 76091392<>76095488 00000100:00100000:3.0:1789760797.451690:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565609472:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.491711:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985885 gen 3 00000008:00010000:2.0:1789760797.491749:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760797.491755:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.491763:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.491784:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760797.491808:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565609472:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760797.491977:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18577, 18577] 00000080:00200000:0.0:1789760797.491989:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760797.491994:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760797.492006:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76095488 i_blocks 148648 00000020:00200000:0.0:1789760797.492010:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760797.492019:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760797.492045:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76095488, count: 4096 00000080:00200000:0.0:1789760797.492051:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76095488, bytes: 4096 00000080:00200000:0.0:1789760797.492058:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760797.492066:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760797.492068:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76095488, 76099584) 0 0 00000080:00200000:0.0:1789760797.492072:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760797.492084:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18578, 18578] 00020000:00200000:0.0:1789760797.492091:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76095488 chunk: [76095488, 76099584) 76099584, 4096 00020000:00200000:0.0:1789760797.492095:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760797.492111:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18578, 18578] 00000008:00010000:0.0:1789760797.492124:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760797.492129:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760797.492138:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76095488, 76099584) append 0 append lockpos 0 00000080:00200000:0.0:1789760797.492149:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76095488=0x4892000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760797.492197:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760797.492200:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76099584 00000020:00010000:0.0:1789760797.492207:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:0.0:1789760797.492211:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.492213:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.492216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760797.492219:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760797.492225:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760797.492231:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760797.492245:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760797.492347:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76095488<>76099584 00000100:00100000:0.0:1789760797.492526:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760797.492542:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565609600:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.514609:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985886 gen 3 00000008:00010000:0.0:1789760797.514678:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760797.514685:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.514693:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.514715:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760797.514738:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565609600:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.514841:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18578, 18578] 00000080:00200000:2.0:1789760797.514854:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.514859:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.514872:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76099584 i_blocks 148656 00000020:00200000:2.0:1789760797.514875:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.514884:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.514909:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76099584, count: 4096 00000080:00200000:2.0:1789760797.514915:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76099584, bytes: 4096 00000080:00200000:2.0:1789760797.514923:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.514929:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.514932:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76099584, 76103680) 0 0 00000080:00200000:2.0:1789760797.514935:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.514948:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18579, 18579] 00020000:00200000:2.0:1789760797.514956:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76099584 chunk: [76099584, 76103680) 76103680, 4096 00020000:00200000:2.0:1789760797.514959:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.514973:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18579, 18579] 00000008:00010000:2.0:1789760797.514990:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.514995:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.515004:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76099584, 76103680) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.515014:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76099584=0x4893000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.515058:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.515061:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76103680 00000020:00010000:2.0:1789760797.515068:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.515072:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.515075:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.515078:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.515081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.515087:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.515093:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.515106:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760797.515200:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 76099584<>76103680 00000100:00100000:2.0:1789760797.515243:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760797.515258:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565609728:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.531794:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985887 gen 3 00000008:00010000:2.0:1789760797.531831:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760797.531837:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.531844:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.531866:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760797.531889:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565609728:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.531996:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18579, 18579] 00000080:00200000:3.0:1789760797.532008:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.532013:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.532024:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76103680 i_blocks 148664 00000020:00200000:3.0:1789760797.532027:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.532035:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.532059:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76103680, count: 4096 00000080:00200000:3.0:1789760797.532066:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76103680, bytes: 4096 00000080:00200000:3.0:1789760797.532075:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.532082:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.532085:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76103680, 76107776) 0 0 00000080:00200000:3.0:1789760797.532088:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.532101:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18580, 18580] 00020000:00200000:3.0:1789760797.532108:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76103680 chunk: [76103680, 76107776) 76107776, 4096 00020000:00200000:3.0:1789760797.532114:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760797.532132:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18580, 18580] 00000008:00010000:3.0:1789760797.532147:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760797.532152:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760797.532161:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76103680, 76107776) append 0 append lockpos 0 00000080:00200000:3.0:1789760797.532173:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76103680=0x4894000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760797.532221:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760797.532224:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76107776 00000020:00010000:3.0:1789760797.532230:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:3.0:1789760797.532234:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.532237:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.532241:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760797.532244:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760797.532251:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760797.532256:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760797.532270:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760797.532380:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 76103680<>76107776 00000100:00100000:3.0:1789760797.532417:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760797.532432:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565609856:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.567171:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985888 gen 3 00000008:00010000:3.0:1789760797.567208:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:3.0:1789760797.567214:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.567223:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.567246:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:3.0:1789760797.567268:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565609856:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760797.567408:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18580, 18580] 00000080:00200000:0.0:1789760797.567421:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760797.567426:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760797.567438:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76107776 i_blocks 148672 00000020:00200000:0.0:1789760797.567441:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760797.567448:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760797.567473:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76107776, count: 4096 00000080:00200000:0.0:1789760797.567480:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76107776, bytes: 4096 00000080:00200000:0.0:1789760797.567496:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760797.567504:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760797.567506:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76107776, 76111872) 0 0 00000080:00200000:0.0:1789760797.567509:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760797.567523:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18581, 18581] 00020000:00200000:0.0:1789760797.567531:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76107776 chunk: [76107776, 76111872) 76111872, 4096 00020000:00200000:0.0:1789760797.567534:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760797.567549:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18581, 18581] 00000008:00010000:0.0:1789760797.567562:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760797.567567:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760797.567575:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76107776, 76111872) append 0 append lockpos 0 00000080:00200000:0.0:1789760797.567586:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76107776=0x4895000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760797.568425:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760797.568430:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76111872 00000020:00010000:0.0:1789760797.568441:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:0.0:1789760797.568445:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.568448:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.568452:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760797.568455:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760797.568463:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760797.568470:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760797.568487:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760797.568588:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76107776<>76111872 00000100:00100000:1.0:1789760797.568677:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565609984:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.584409:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985889 gen 3 00000008:00010000:0.0:1789760797.584444:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760797.584450:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.584459:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.584539:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760797.584561:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565609984:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.588204:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18581, 18581] 00000080:00200000:3.0:1789760797.588224:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.588229:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.589802:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76111872 i_blocks 148680 00000020:00200000:2.0:1789760797.589812:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.589828:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.589856:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76111872, count: 4096 00000080:00200000:2.0:1789760797.589863:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76111872, bytes: 4096 00000080:00200000:2.0:1789760797.589875:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.589882:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.589884:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76111872, 76115968) 0 0 00000080:00200000:2.0:1789760797.589888:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.589902:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18582, 18582] 00020000:00200000:2.0:1789760797.589909:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76111872 chunk: [76111872, 76115968) 76115968, 4096 00020000:00200000:2.0:1789760797.589913:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.589930:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18582, 18582] 00000008:00010000:2.0:1789760797.589943:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.589949:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.589957:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76111872, 76115968) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.589968:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76111872=0x4896000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.590022:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.590024:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76115968 00000020:00010000:2.0:1789760797.590031:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.590034:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.590036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e248/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.590039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.590042:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.590048:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.590054:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.590098:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760797.590200:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 76111872<>76115968 00000100:00100000:2.0:1789760797.590250:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760797.590273:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565610112:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.602033:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985890 gen 3 00000008:00010000:2.0:1789760797.602076:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760797.602085:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.602093:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.602119:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760797.602145:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565610112:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760797.602258:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18582, 18582] 00000080:00200000:0.0:1789760797.602275:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760797.602283:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760797.602295:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76115968 i_blocks 148688 00000020:00200000:0.0:1789760797.602302:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760797.602312:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760797.602340:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76115968, count: 4096 00000080:00200000:0.0:1789760797.602349:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76115968, bytes: 4096 00000080:00200000:0.0:1789760797.602358:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760797.602373:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760797.602375:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76115968, 76120064) 0 0 00000080:00200000:0.0:1789760797.602378:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760797.602396:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18583, 18583] 00020000:00200000:0.0:1789760797.602410:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76115968 chunk: [76115968, 76120064) 76120064, 4096 00020000:00200000:0.0:1789760797.602414:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760797.602430:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18583, 18583] 00000008:00010000:0.0:1789760797.602443:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760797.602448:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760797.602457:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76115968, 76120064) append 0 append lockpos 0 00000080:00200000:0.0:1789760797.602468:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76115968=0x4897000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760797.602512:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760797.602515:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76120064 00000020:00010000:0.0:1789760797.602522:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:0.0:1789760797.602525:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.602528:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e248/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.602531:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760797.602533:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760797.602539:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760797.602545:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760797.602558:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760797.602693:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76115968<>76120064 00000100:00100000:0.0:1789760797.602724:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760797.602736:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565610240:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.619058:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985891 gen 3 00000008:00010000:0.0:1789760797.619097:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760797.619103:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.619111:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.619135:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760797.619157:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565610240:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.619258:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18583, 18583] 00000080:00200000:2.0:1789760797.619271:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.619276:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.619288:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76120064 i_blocks 148696 00000020:00200000:2.0:1789760797.619291:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.619300:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.619326:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76120064, count: 4096 00000080:00200000:2.0:1789760797.619333:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76120064, bytes: 4096 00000080:00200000:2.0:1789760797.619341:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.619348:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.619350:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76120064, 76124160) 0 0 00000080:00200000:2.0:1789760797.619354:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.619381:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18584, 18584] 00020000:00200000:2.0:1789760797.619388:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76120064 chunk: [76120064, 76124160) 76124160, 4096 00020000:00200000:2.0:1789760797.619391:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.619408:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18584, 18584] 00000008:00010000:2.0:1789760797.619420:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.619425:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.619433:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76120064, 76124160) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.619444:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76120064=0x4898000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.619489:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.619492:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76124160 00000020:00010000:2.0:1789760797.619498:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.619502:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.619505:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e248/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.619508:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420eb78 (ffff93f84354e998/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.619510:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e998/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.619516:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.619522:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.619537:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760797.619806:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 76120064<>76124160 00000100:00100000:2.0:1789760797.619855:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565610368:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.641548:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985892 gen 3 00000008:00010000:2.0:1789760797.641586:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760797.641592:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.641767:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.641786:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760797.641809:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565610368:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.641902:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18584, 18584] 00000080:00200000:3.0:1789760797.641915:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.641920:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.641932:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76124160 i_blocks 148704 00000020:00200000:3.0:1789760797.641935:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.641944:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.641991:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76124160, count: 4096 00000080:00200000:3.0:1789760797.641998:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76124160, bytes: 4096 00000080:00200000:3.0:1789760797.642005:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.642012:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.642014:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76124160, 76128256) 0 0 00000080:00200000:3.0:1789760797.642018:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.642030:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18585, 18585] 00020000:00200000:3.0:1789760797.642037:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76124160 chunk: [76124160, 76128256) 76128256, 4096 00020000:00200000:3.0:1789760797.642041:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760797.642056:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18585, 18585] 00000008:00010000:3.0:1789760797.642068:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760797.642074:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760797.642084:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76124160, 76128256) append 0 append lockpos 0 00000080:00200000:3.0:1789760797.642094:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76124160=0x4899000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760797.642139:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760797.642142:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76128256 00000020:00010000:3.0:1789760797.642149:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_release():245 00000020:00010000:3.0:1789760797.642153:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.642155:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.642159:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760797.642162:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e248/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760797.642168:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760797.642174:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760797.642188:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760797.642278:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d66300 - object 0x240000400:5 offset 76124160<>76128256 00000100:00100000:2.0:1789760797.642427:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760797.642441:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565610496:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.655786:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985893 gen 3 00000008:00010000:2.0:1789760797.655824:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:2.0:1789760797.655831:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.655840:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.655913:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:2.0:1789760797.655938:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565610496:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.658627:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18585, 18585] 00000080:00200000:3.0:1789760797.658657:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.658663:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.658820:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76128256 i_blocks 148712 00000020:00200000:2.0:1789760797.658826:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.658840:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.658865:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76128256, count: 4096 00000080:00200000:2.0:1789760797.658873:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76128256, bytes: 4096 00000080:00200000:2.0:1789760797.658884:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.658891:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.658894:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76128256, 76132352) 0 0 00000080:00200000:2.0:1789760797.658898:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.658912:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18586, 18586] 00020000:00200000:2.0:1789760797.658919:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76128256 chunk: [76128256, 76132352) 76132352, 4096 00020000:00200000:2.0:1789760797.658923:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.658940:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18586, 18586] 00000008:00010000:2.0:1789760797.658955:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.658959:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.658968:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76128256, 76132352) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.658979:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76128256=0x489a000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.659029:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.659032:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76132352 00000020:00010000:2.0:1789760797.659040:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.659044:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.659046:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e368/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.659049:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.659051:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e248/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.659058:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.659063:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.659108:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760797.659210:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 76128256<>76132352 00000100:00100000:2.0:1789760797.659257:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760797.659276:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565610624:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.691248:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985894 gen 3 00000008:00010000:2.0:1789760797.691285:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760797.691291:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.691300:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.691326:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760797.691350:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565610624:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760797.691413:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18586, 18586] 00000080:00200000:0.0:1789760797.691426:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760797.691432:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760797.691443:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76132352 i_blocks 148720 00000020:00200000:0.0:1789760797.691447:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760797.691455:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760797.691481:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76132352, count: 4096 00000080:00200000:0.0:1789760797.691488:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76132352, bytes: 4096 00000080:00200000:0.0:1789760797.691495:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760797.691502:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760797.691504:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76132352, 76136448) 0 0 00000080:00200000:0.0:1789760797.691509:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760797.691522:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18587, 18587] 00020000:00200000:0.0:1789760797.691529:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76132352 chunk: [76132352, 76136448) 76136448, 4096 00020000:00200000:0.0:1789760797.691533:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760797.691549:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18587, 18587] 00000008:00010000:0.0:1789760797.691562:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760797.691566:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760797.691575:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76132352, 76136448) append 0 append lockpos 0 00000080:00200000:0.0:1789760797.691714:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76132352=0x489b000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760797.691801:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760797.691805:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76136448 00000020:00010000:0.0:1789760797.691813:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_release():245 00000020:00010000:0.0:1789760797.691816:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.691819:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e368/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.691822:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760797.691826:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e248/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760797.691832:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760797.691839:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760797.691851:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760797.691969:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76132352<>76136448 00000100:00100000:1.0:1789760797.692142:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565610752:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.705680:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985895 gen 3 00000008:00010000:0.0:1789760797.705836:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760797.705843:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.705852:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.705969:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18587, 18587] 00000080:00200000:2.0:1789760797.705984:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.705989:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.706001:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76136448 i_blocks 148728 00000020:00200000:2.0:1789760797.706004:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.706013:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.706040:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76136448, count: 4096 00000080:00200000:2.0:1789760797.706047:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76136448, bytes: 4096 00000080:00200000:2.0:1789760797.706055:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.706063:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.706065:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76136448, 76140544) 0 0 00000080:00200000:2.0:1789760797.706069:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.706082:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18588, 18588] 00020000:00200000:2.0:1789760797.706089:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76136448 chunk: [76136448, 76140544) 76140544, 4096 00020000:00200000:2.0:1789760797.706093:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.706109:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18588, 18588] 00000008:00010000:2.0:1789760797.706122:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.706127:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.706136:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76136448, 76140544) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.706147:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76136448=0x489c000, pages 1 (max 131072), lockless, parallel 00000008:00010000:0.0:1789760797.706199:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760797.706224:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565610752:192.168.203.147@tcp:4:dd.0 00000020:00200000:2.0:1789760797.708831:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.708840:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76140544 00000020:00010000:2.0:1789760797.708855:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.708860:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.708865:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e368/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.708868:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.708944:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e248/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.708957:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.708965:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.708999:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760797.709111:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 76136448<>76140544 00000100:00100000:3.0:1789760797.709201:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565610880:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.724421:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985896 gen 3 00000008:00010000:2.0:1789760797.724460:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760797.724466:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.724474:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.724556:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760797.724578:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565610880:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760797.724660:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18588, 18588] 00000080:00200000:0.0:1789760797.724739:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760797.724746:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760797.724757:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76140544 i_blocks 148736 00000020:00200000:0.0:1789760797.724762:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760797.724770:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760797.724795:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76140544, count: 4096 00000080:00200000:0.0:1789760797.724802:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76140544, bytes: 4096 00000080:00200000:0.0:1789760797.724810:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760797.724817:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760797.724819:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76140544, 76144640) 0 0 00000080:00200000:0.0:1789760797.724823:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760797.724835:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18589, 18589] 00020000:00200000:0.0:1789760797.724841:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76140544 chunk: [76140544, 76144640) 76144640, 4096 00020000:00200000:0.0:1789760797.724845:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760797.724859:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18589, 18589] 00000008:00010000:0.0:1789760797.724872:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760797.724877:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760797.724886:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76140544, 76144640) append 0 append lockpos 0 00000080:00200000:0.0:1789760797.724897:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76140544=0x489d000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760797.725068:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760797.725071:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76144640 00000020:00010000:0.0:1789760797.725077:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_release():245 00000020:00010000:0.0:1789760797.725081:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.725083:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84354e368/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.725086:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760797.725089:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e248/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760797.725095:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760797.725101:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760797.725115:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760797.725203:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76140544<>76144640 00000100:00100000:1.0:1789760797.725267:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565611008:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.737149:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985897 gen 3 00000008:00010000:0.0:1789760797.737185:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760797.737191:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.737200:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.737219:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760797.737240:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565611008:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.737393:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18589, 18589] 00000080:00200000:2.0:1789760797.737405:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.737409:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.737420:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76144640 i_blocks 148744 00000020:00200000:2.0:1789760797.737513:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.737524:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.737547:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76144640, count: 4096 00000080:00200000:2.0:1789760797.737553:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76144640, bytes: 4096 00000080:00200000:2.0:1789760797.737561:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.737569:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.737571:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76144640, 76148736) 0 0 00000080:00200000:2.0:1789760797.737575:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.737587:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18590, 18590] 00020000:00200000:2.0:1789760797.737593:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76144640 chunk: [76144640, 76148736) 76148736, 4096 00020000:00200000:2.0:1789760797.737597:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.737610:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18590, 18590] 00000008:00010000:2.0:1789760797.737622:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.737626:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.737634:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76144640, 76148736) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.737767:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76144640=0x489e000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.737806:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.737808:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76148736 00000020:00010000:2.0:1789760797.737814:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.737817:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.737820:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84354e368/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.737823:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e5d8 (ffff93f84354e248/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.737825:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e248/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.737831:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.737836:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.737848:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760797.737931:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 76144640<>76148736 00000100:00100000:3.0:1789760797.738045:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565611136:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.759893:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985898 gen 3 00000008:00010000:2.0:1789760797.759928:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:2.0:1789760797.759935:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.759945:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.759977:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:2.0:1789760797.760002:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565611136:192.168.203.147@tcp:4:dd.0 00000080:00200000:3.0:1789760797.760268:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18590, 18590] 00000080:00200000:3.0:1789760797.760282:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1789760797.760288:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1789760797.760300:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76148736 i_blocks 148752 00000020:00200000:3.0:1789760797.760379:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1789760797.760388:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:3.0:1789760797.760436:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76148736, count: 4096 00000080:00200000:3.0:1789760797.760444:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76148736, bytes: 4096 00000080:00200000:3.0:1789760797.760452:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:3.0:1789760797.760460:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:3.0:1789760797.760462:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76148736, 76152832) 0 0 00000080:00200000:3.0:1789760797.760466:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:3.0:1789760797.760479:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18591, 18591] 00020000:00200000:3.0:1789760797.760485:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76148736 chunk: [76148736, 76152832) 76152832, 4096 00020000:00200000:3.0:1789760797.760489:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760797.760504:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18591, 18591] 00000008:00010000:3.0:1789760797.760517:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:3.0:1789760797.760522:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f84567e730, osc lock ffff93f8457e71d0, flags 0x0 00000080:00200000:3.0:1789760797.760530:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76148736, 76152832) append 0 append lockpos 0 00000080:00200000:3.0:1789760797.760540:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76148736=0x489f000, pages 1 (max 131072), lockless, parallel 00000020:00200000:3.0:1789760797.760582:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1789760797.760585:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76152832 00000020:00010000:3.0:1789760797.760592:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_release():245 00000020:00010000:3.0:1789760797.760596:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.760598:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84567e730 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:3.0:1789760797.760601:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_fini():72 00000020:00010000:3.0:1789760797.760604:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84567e730 (ffff93f84354e368/1) at cl_lock_fini():72 00000080:00200000:3.0:1789760797.760610:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:3.0:1789760797.760614:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:3.0:1789760797.760627:0:254:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f85a362c00 00000008:00100000:3.0:1789760797.760755:0:254:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844d65180 - object 0x240000400:5 offset 76148736<>76152832 00000100:00100000:3.0:1789760797.760801:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760797.760816:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565611264:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.781473:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985899 gen 3 00000008:00010000:3.0:1789760797.781528:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f85a362c00 00000020:00200000:3.0:1789760797.781535:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.781545:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.781569:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f85a362c00 00000100:00100000:3.0:1789760797.781595:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565611264:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.781604:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18591, 18591] 00000080:00200000:2.0:1789760797.781620:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.781625:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.781637:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76152832 i_blocks 148760 00000020:00200000:2.0:1789760797.781681:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.781692:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.781718:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76152832, count: 4096 00000080:00200000:2.0:1789760797.781724:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76152832, bytes: 4096 00000080:00200000:2.0:1789760797.781733:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.781750:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.781753:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76152832, 76156928) 0 0 00000080:00200000:2.0:1789760797.781757:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.781770:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18592, 18592] 00020000:00200000:2.0:1789760797.781777:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76152832 chunk: [76152832, 76156928) 76156928, 4096 00020000:00200000:2.0:1789760797.781781:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.781796:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18592, 18592] 00000008:00010000:2.0:1789760797.781809:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.781827:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.781837:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76152832, 76156928) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.781848:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76152832=0x48a0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.781889:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.781892:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76156928 00000020:00010000:2.0:1789760797.781900:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.781904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.781906:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.781915:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.781918:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e368/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.781925:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.781931:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.781945:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fb40 00000008:00100000:2.0:1789760797.782048:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c1880 - object 0x240000400:5 offset 76152832<>76156928 00000100:00100000:3.0:1789760797.782241:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565611392:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.793435:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985900 gen 3 00000008:00010000:3.0:1789760797.793472:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fb40 00000020:00200000:3.0:1789760797.793478:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.793487:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.793511:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fb40 00000100:00100000:3.0:1789760797.793537:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565611392:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.794458:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18592, 18592] 00000080:00200000:2.0:1789760797.794472:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.794476:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.794487:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76156928 i_blocks 148768 00000020:00200000:2.0:1789760797.794491:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.794499:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.794523:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76156928, count: 4096 00000080:00200000:2.0:1789760797.794529:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76156928, bytes: 4096 00000080:00200000:2.0:1789760797.794538:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.794545:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.794547:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76156928, 76161024) 0 0 00000080:00200000:2.0:1789760797.794551:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.794563:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18593, 18593] 00020000:00200000:2.0:1789760797.794569:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76156928 chunk: [76156928, 76161024) 76161024, 4096 00020000:00200000:2.0:1789760797.794573:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.794588:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18593, 18593] 00000008:00010000:2.0:1789760797.794600:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.794605:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.794614:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76156928, 76161024) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.794624:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76156928=0x48a1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.794690:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.794693:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76161024 00000020:00010000:2.0:1789760797.794699:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.794702:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.794705:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.794708:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.794710:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e368/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.794716:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.794722:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.794735:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f900 00000008:00100000:2.0:1789760797.794844:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2d80 - object 0x240000400:5 offset 76156928<>76161024 00000100:00100000:3.0:1789760797.795222:0:2360:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760797.795241:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565611520:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.811614:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985901 gen 3 00000008:00010000:3.0:1789760797.812044:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f900 00000020:00200000:3.0:1789760797.812051:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.812060:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.812180:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f900 00000100:00100000:3.0:1789760797.812206:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565611520:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.812674:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18593, 18593] 00000080:00200000:2.0:1789760797.812688:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.812694:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.812705:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76161024 i_blocks 148776 00000020:00200000:2.0:1789760797.812709:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.812717:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.812962:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76161024, count: 4096 00000080:00200000:2.0:1789760797.812971:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76161024, bytes: 4096 00000080:00200000:2.0:1789760797.812981:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.812989:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.812990:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76161024, 76165120) 0 0 00000080:00200000:2.0:1789760797.812994:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.813007:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18594, 18594] 00020000:00200000:2.0:1789760797.813015:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76161024 chunk: [76161024, 76165120) 76165120, 4096 00020000:00200000:2.0:1789760797.813019:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.813033:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18594, 18594] 00000008:00010000:2.0:1789760797.813047:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.813052:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.813142:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76161024, 76165120) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.813154:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76161024=0x48a2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.813204:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.813207:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76165120 00000020:00010000:2.0:1789760797.813213:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.813216:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.813220:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.813222:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.813225:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e368/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.813231:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.813236:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.813253:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354f180 00000008:00100000:2.0:1789760797.813348:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c2a00 - object 0x240000400:5 offset 76161024<>76165120 00000100:00100000:3.0:1789760797.813445:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565611648:192.168.203.147@tcp:4:dd.0 00000100:00100000:3.0:1789760797.832163:0:2360:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985902 gen 3 00000008:00010000:3.0:1789760797.832203:0:2360:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354f180 00000020:00200000:3.0:1789760797.832211:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:3.0:1789760797.832221:0:2360:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1789760797.832245:0:2360:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354f180 00000100:00100000:3.0:1789760797.832272:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565611648:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760797.832508:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18594, 18594] 00000080:00200000:2.0:1789760797.832520:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760797.832524:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760797.832536:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76165120 i_blocks 148784 00000020:00200000:2.0:1789760797.832539:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760797.832547:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760797.832571:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76165120, count: 4096 00000080:00200000:2.0:1789760797.832577:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76165120, bytes: 4096 00000080:00200000:2.0:1789760797.832584:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760797.832590:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760797.832593:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76165120, 76169216) 0 0 00000080:00200000:2.0:1789760797.832596:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760797.832606:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18595, 18595] 00020000:00200000:2.0:1789760797.832613:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76165120 chunk: [76165120, 76169216) 76169216, 4096 00020000:00200000:2.0:1789760797.832616:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760797.832628:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18595, 18595] 00000008:00010000:2.0:1789760797.832639:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760797.832672:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760797.832682:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76165120, 76169216) append 0 append lockpos 0 00000080:00200000:2.0:1789760797.832692:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76165120=0x48a3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760797.832734:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760797.832737:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76169216 00000020:00010000:2.0:1789760797.832744:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_release():245 00000020:00010000:2.0:1789760797.832747:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.832750:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760797.832752:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760797.832756:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84354e368/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760797.832762:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760797.832768:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760797.832782:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760797.832871:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 76165120<>76169216 00000100:00100000:3.0:1789760797.833592:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565611776:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760797.849475:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985903 gen 3 00000008:00010000:2.0:1789760797.850151:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:2.0:1789760797.850158:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760797.850166:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760797.850254:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:2.0:1789760797.850280:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565611776:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760797.851778:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18595, 18595] 00000080:00200000:0.0:1789760797.851796:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760797.851802:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760797.851814:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76169216 i_blocks 148792 00000020:00200000:0.0:1789760797.851819:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760797.851829:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760797.851855:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76169216, count: 4096 00000080:00200000:0.0:1789760797.851861:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76169216, bytes: 4096 00000080:00200000:0.0:1789760797.851872:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760797.851881:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760797.851883:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76169216, 76173312) 0 0 00000080:00200000:0.0:1789760797.851887:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760797.851900:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18596, 18596] 00020000:00200000:0.0:1789760797.851907:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76169216 chunk: [76169216, 76173312) 76173312, 4096 00020000:00200000:0.0:1789760797.851911:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760797.851925:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18596, 18596] 00000008:00010000:0.0:1789760797.852458:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760797.852465:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760797.852476:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76169216, 76173312) append 0 append lockpos 0 00000080:00200000:0.0:1789760797.852521:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76169216=0x48a4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760797.852591:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760797.852594:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76173312 00000020:00010000:0.0:1789760797.852601:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_release():245 00000020:00010000:0.0:1789760797.852604:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.852612:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418a098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.852615:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f84420e7b8 (ffff93f84354e368/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760797.852617:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84354e368/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760797.852624:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760797.853066:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760797.853091:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760797.853194:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76169216<>76173312 00000100:00100000:1.0:1789760797.853268:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565611904:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.872316:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985904 gen 3 00000008:00010000:0.0:1789760797.872350:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760797.872356:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.872376:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.872396:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760797.872419:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565611904:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.872423:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18596, 18596] 00000080:00200000:1.0:1789760797.872438:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.872443:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.872454:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76173312 i_blocks 148800 00000020:00200000:1.0:1789760797.872458:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.872470:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760797.872589:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76173312, count: 4096 00000080:00200000:0.0:1789760797.872598:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76173312, bytes: 4096 00000080:00200000:0.0:1789760797.872607:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760797.872614:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760797.872616:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76173312, 76177408) 0 0 00000080:00200000:0.0:1789760797.872621:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760797.872634:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18597, 18597] 00020000:00200000:0.0:1789760797.872641:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76173312 chunk: [76173312, 76177408) 76177408, 4096 00020000:00200000:0.0:1789760797.872820:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760797.872836:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18597, 18597] 00000008:00010000:0.0:1789760797.872850:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760797.872855:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760797.872863:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76173312, 76177408) append 0 append lockpos 0 00000080:00200000:0.0:1789760797.872874:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76173312=0x48a5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760797.872913:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760797.872915:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76177408 00000020:00010000:0.0:1789760797.872932:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760797.872936:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.872938:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760797.872941:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760797.872944:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760797.872951:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760797.872956:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760797.872969:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760797.873220:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76173312<>76177408 00000100:00100000:0.0:1789760797.873571:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760797.873588:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612032:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.892090:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985905 gen 3 00000008:00010000:0.0:1789760797.892124:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760797.892129:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.892137:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.892154:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760797.892175:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612032:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.892257:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18597, 18597] 00000080:00200000:1.0:1789760797.892269:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.892274:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.892284:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76177408 i_blocks 148808 00000020:00200000:1.0:1789760797.892305:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.892313:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.892337:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76177408, count: 4096 00000080:00200000:1.0:1789760797.892343:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76177408, bytes: 4096 00000080:00200000:1.0:1789760797.892350:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.892357:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.892372:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76177408, 76181504) 0 0 00000080:00200000:1.0:1789760797.892376:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.892387:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18598, 18598] 00020000:00200000:1.0:1789760797.892394:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76177408 chunk: [76177408, 76181504) 76181504, 4096 00020000:00200000:1.0:1789760797.892398:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.892411:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18598, 18598] 00000008:00010000:1.0:1789760797.892423:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.892428:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.892436:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76177408, 76181504) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.892445:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76177408=0x48a6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.892487:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.892490:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76181504 00000020:00010000:1.0:1789760797.892497:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.892501:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.892504:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.892506:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.892509:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.892515:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.892520:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.892536:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760797.892624:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 76177408<>76181504 00000100:00100000:1.0:1789760797.892687:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760797.892699:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565612160:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.907010:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985906 gen 3 00000008:00010000:0.0:1789760797.907042:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760797.907048:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.907056:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.907075:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760797.907096:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565612160:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.907176:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18598, 18598] 00000080:00200000:1.0:1789760797.907188:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.907193:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.907203:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76181504 i_blocks 148816 00000020:00200000:1.0:1789760797.907207:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.907215:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.907238:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76181504, count: 4096 00000080:00200000:1.0:1789760797.907244:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76181504, bytes: 4096 00000080:00200000:1.0:1789760797.907250:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.907267:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.907269:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76181504, 76185600) 0 0 00000080:00200000:1.0:1789760797.907272:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.907284:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18599, 18599] 00020000:00200000:1.0:1789760797.907290:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76181504 chunk: [76181504, 76185600) 76185600, 4096 00020000:00200000:1.0:1789760797.907294:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.907307:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18599, 18599] 00000008:00010000:1.0:1789760797.907319:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.907323:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.907331:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76181504, 76185600) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.907342:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76181504=0x48a7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.907400:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.907402:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76185600 00000020:00010000:1.0:1789760797.907408:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.907411:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.907414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.907417:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.907420:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.907426:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.907430:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.907442:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760797.907538:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 76181504<>76185600 00000100:00100000:0.0:1789760797.907609:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760797.907623:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612288:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.923757:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985907 gen 3 00000008:00010000:0.0:1789760797.923791:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760797.923797:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.923804:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.923822:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760797.923843:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612288:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.923925:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18599, 18599] 00000080:00200000:1.0:1789760797.923938:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.923943:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.923955:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76185600 i_blocks 148824 00000020:00200000:1.0:1789760797.923958:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.923966:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.923990:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76185600, count: 4096 00000080:00200000:1.0:1789760797.923996:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76185600, bytes: 4096 00000080:00200000:1.0:1789760797.924004:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.924011:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.924013:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76185600, 76189696) 0 0 00000080:00200000:1.0:1789760797.924017:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.924029:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18600, 18600] 00020000:00200000:1.0:1789760797.924036:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76185600 chunk: [76185600, 76189696) 76189696, 4096 00020000:00200000:1.0:1789760797.924040:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.924053:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18600, 18600] 00000008:00010000:1.0:1789760797.924066:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.924071:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.924080:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76185600, 76189696) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.924090:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76185600=0x48a8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.924133:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.924135:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76189696 00000020:00010000:1.0:1789760797.924142:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.924145:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.924148:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.924151:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.924154:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.924160:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.924165:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.924178:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760797.924274:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9880 - object 0x240000400:5 offset 76185600<>76189696 00000100:00100000:0.0:1789760797.924334:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612416:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.937603:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985908 gen 3 00000008:00010000:0.0:1789760797.937640:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760797.937672:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.937680:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.937741:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760797.937763:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612416:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.942152:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18600, 18600] 00000080:00200000:1.0:1789760797.942172:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.942178:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.942192:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76189696 i_blocks 148832 00000020:00200000:1.0:1789760797.942196:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.942209:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.942234:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76189696, count: 4096 00000080:00200000:1.0:1789760797.942241:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76189696, bytes: 4096 00000080:00200000:1.0:1789760797.942251:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.942259:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.942261:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76189696, 76193792) 0 0 00000080:00200000:1.0:1789760797.942264:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.942276:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18601, 18601] 00020000:00200000:1.0:1789760797.942283:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76189696 chunk: [76189696, 76193792) 76193792, 4096 00020000:00200000:1.0:1789760797.942287:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.942301:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18601, 18601] 00000008:00010000:1.0:1789760797.942314:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.942320:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.942329:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76189696, 76193792) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.942339:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76189696=0x48a9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.942404:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.942407:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76193792 00000020:00010000:1.0:1789760797.942414:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.942417:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.942419:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.942423:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.942425:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.942432:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.942438:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.942461:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760797.942567:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9f80 - object 0x240000400:5 offset 76189696<>76193792 00000100:00100000:0.0:1789760797.942803:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565612544:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.955461:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985909 gen 3 00000008:00010000:0.0:1789760797.955498:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760797.955505:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.955514:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.955533:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760797.955554:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565612544:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.955722:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18601, 18601] 00000080:00200000:1.0:1789760797.955735:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.955740:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.955752:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76193792 i_blocks 148840 00000020:00200000:1.0:1789760797.955755:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.955763:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.955789:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76193792, count: 4096 00000080:00200000:1.0:1789760797.955795:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76193792, bytes: 4096 00000080:00200000:1.0:1789760797.955802:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.955809:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.955811:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76193792, 76197888) 0 0 00000080:00200000:1.0:1789760797.955815:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.955827:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18602, 18602] 00020000:00200000:1.0:1789760797.955834:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76193792 chunk: [76193792, 76197888) 76197888, 4096 00020000:00200000:1.0:1789760797.955838:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.955851:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18602, 18602] 00000008:00010000:1.0:1789760797.955865:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.955870:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.955879:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76193792, 76197888) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.955889:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76193792=0x48aa000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.955934:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.955937:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76197888 00000020:00010000:1.0:1789760797.955943:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.955947:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.955949:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.955952:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.955955:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.955960:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.955965:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.955980:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760797.956076:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9180 - object 0x240000400:5 offset 76193792<>76197888 00000100:00100000:0.0:1789760797.956244:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612672:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.966284:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985910 gen 3 00000008:00010000:0.0:1789760797.966334:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760797.966341:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.966349:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.966376:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760797.966400:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612672:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.966548:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18602, 18602] 00000080:00200000:1.0:1789760797.966560:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.966565:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.966577:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76197888 i_blocks 148848 00000020:00200000:1.0:1789760797.966580:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.966588:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.966615:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76197888, count: 4096 00000080:00200000:1.0:1789760797.966622:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76197888, bytes: 4096 00000080:00200000:1.0:1789760797.966629:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.966636:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.966638:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76197888, 76201984) 0 0 00000080:00200000:1.0:1789760797.966677:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.966691:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18603, 18603] 00020000:00200000:1.0:1789760797.966698:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76197888 chunk: [76197888, 76201984) 76201984, 4096 00020000:00200000:1.0:1789760797.966702:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.966726:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18603, 18603] 00000008:00010000:1.0:1789760797.966741:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.966746:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.966755:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76197888, 76201984) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.966766:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76197888=0x48ab000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.966811:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.966814:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76201984 00000020:00010000:1.0:1789760797.966821:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.966824:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.966827:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.966829:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.966832:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.966838:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.966843:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.966857:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760797.966953:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca300 - object 0x240000400:5 offset 76197888<>76201984 00000100:00100000:0.0:1789760797.967103:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760797.967118:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612800:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.982954:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985911 gen 3 00000008:00010000:0.0:1789760797.982990:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760797.982996:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.983004:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.983025:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760797.983050:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612800:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.983198:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18603, 18603] 00000080:00200000:1.0:1789760797.983211:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.983216:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.983228:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76201984 i_blocks 148856 00000020:00200000:1.0:1789760797.983232:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.983240:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.983264:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76201984, count: 4096 00000080:00200000:1.0:1789760797.983270:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76201984, bytes: 4096 00000080:00200000:1.0:1789760797.983278:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.983286:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.983288:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76201984, 76206080) 0 0 00000080:00200000:1.0:1789760797.983291:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.983316:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18604, 18604] 00020000:00200000:1.0:1789760797.983324:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76201984 chunk: [76201984, 76206080) 76206080, 4096 00020000:00200000:1.0:1789760797.983328:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.983343:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18604, 18604] 00000008:00010000:1.0:1789760797.983357:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.983374:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.983385:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76201984, 76206080) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.983395:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76201984=0x48ac000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.983439:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.983441:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76206080 00000020:00010000:1.0:1789760797.983448:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.983451:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.983453:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.983457:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.983459:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.983465:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.983470:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.983485:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760797.983579:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8000 - object 0x240000400:5 offset 76201984<>76206080 00000100:00100000:0.0:1789760797.983740:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612928:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760797.994175:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985912 gen 3 00000008:00010000:0.0:1789760797.994212:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760797.994218:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760797.994227:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760797.994249:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760797.994272:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565612928:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760797.994684:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18604, 18604] 00000080:00200000:1.0:1789760797.994696:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760797.994701:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760797.994714:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76206080 i_blocks 148864 00000020:00200000:1.0:1789760797.994717:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760797.994726:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760797.994751:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76206080, count: 4096 00000080:00200000:1.0:1789760797.994757:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76206080, bytes: 4096 00000080:00200000:1.0:1789760797.994765:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760797.994772:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760797.994774:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76206080, 76210176) 0 0 00000080:00200000:1.0:1789760797.994778:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760797.994790:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18605, 18605] 00020000:00200000:1.0:1789760797.994797:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76206080 chunk: [76206080, 76210176) 76210176, 4096 00020000:00200000:1.0:1789760797.994800:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760797.994814:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18605, 18605] 00000008:00010000:1.0:1789760797.994826:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760797.994831:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760797.994839:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76206080, 76210176) append 0 append lockpos 0 00000080:00200000:1.0:1789760797.994850:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76206080=0x48ad000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760797.994894:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760797.994898:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76210176 00000020:00010000:1.0:1789760797.994905:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760797.994908:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.994911:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760797.994914:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760797.994916:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760797.994923:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760797.994929:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760797.994944:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760797.995037:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccaa00 - object 0x240000400:5 offset 76206080<>76210176 00000100:00100000:0.0:1789760797.995234:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760797.995252:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613056:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.012474:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985913 gen 3 00000008:00010000:0.0:1789760798.012517:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760798.012525:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.012534:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.012555:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760798.012582:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613056:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.012681:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18605, 18605] 00000080:00200000:1.0:1789760798.012694:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.012698:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.012710:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76210176 i_blocks 148872 00000020:00200000:1.0:1789760798.012723:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.012731:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.012755:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76210176, count: 4096 00000080:00200000:1.0:1789760798.012761:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76210176, bytes: 4096 00000080:00200000:1.0:1789760798.012769:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.012776:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.012778:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76210176, 76214272) 0 0 00000080:00200000:1.0:1789760798.012781:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.012796:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18606, 18606] 00020000:00200000:1.0:1789760798.012803:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76210176 chunk: [76210176, 76214272) 76214272, 4096 00020000:00200000:1.0:1789760798.012806:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.012821:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18606, 18606] 00000008:00010000:1.0:1789760798.012835:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.012839:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.012849:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76210176, 76214272) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.012861:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76210176=0x48ae000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.012904:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.012906:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76214272 00000020:00010000:1.0:1789760798.012913:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.012916:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.012919:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.012922:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.012925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.012930:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.012936:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.012949:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:1.0:1789760798.013048:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb100 - object 0x240000400:5 offset 76210176<>76214272 00000100:00100000:0.0:1789760798.013208:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613184:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.031123:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985914 gen 3 00000008:00010000:0.0:1789760798.031167:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760798.031174:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.031182:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.031239:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760798.031263:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613184:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.031392:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18606, 18606] 00000080:00200000:1.0:1789760798.031405:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.031410:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.031586:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76214272 i_blocks 148880 00000020:00200000:1.0:1789760798.031592:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.031600:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.031625:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76214272, count: 4096 00000080:00200000:1.0:1789760798.031632:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76214272, bytes: 4096 00000080:00200000:1.0:1789760798.031639:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.031785:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.031787:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76214272, 76218368) 0 0 00000080:00200000:1.0:1789760798.031791:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.031804:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18607, 18607] 00020000:00200000:1.0:1789760798.031810:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76214272 chunk: [76214272, 76218368) 76218368, 4096 00020000:00200000:1.0:1789760798.031814:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.031829:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18607, 18607] 00000008:00010000:1.0:1789760798.031842:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.031846:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.031855:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76214272, 76218368) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.031866:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76214272=0x48af000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.031909:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.031912:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76218368 00000020:00010000:1.0:1789760798.031919:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.031923:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.031925:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.031928:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.031931:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.031936:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.031941:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.031955:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760798.032050:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccbb80 - object 0x240000400:5 offset 76214272<>76218368 00000100:00100000:0.0:1789760798.032114:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565613312:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.045130:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985915 gen 3 00000008:00010000:0.0:1789760798.045163:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760798.045169:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.045177:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.045195:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760798.045215:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565613312:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.045307:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18607, 18607] 00000080:00200000:1.0:1789760798.045319:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.045323:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.045334:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76218368 i_blocks 148888 00000020:00200000:1.0:1789760798.045337:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.045344:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.045390:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76218368, count: 4096 00000080:00200000:1.0:1789760798.045397:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76218368, bytes: 4096 00000080:00200000:1.0:1789760798.045404:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.045411:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.045414:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76218368, 76222464) 0 0 00000080:00200000:1.0:1789760798.045418:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.045430:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18608, 18608] 00020000:00200000:1.0:1789760798.045436:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76218368 chunk: [76218368, 76222464) 76222464, 4096 00020000:00200000:1.0:1789760798.045440:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.045454:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18608, 18608] 00000008:00010000:1.0:1789760798.045465:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.045470:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.045477:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76218368, 76222464) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.045487:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76218368=0x48b0000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.045534:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.045537:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76222464 00000020:00010000:1.0:1789760798.045544:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.045547:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.045549:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.045553:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.045555:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.045561:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.045565:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.045580:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b9c0 00000008:00100000:1.0:1789760798.045726:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9c00 - object 0x240000400:5 offset 76218368<>76222464 00000100:00100000:0.0:1789760798.045990:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613440:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.061357:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985916 gen 3 00000008:00010000:0.0:1789760798.061402:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b9c0 00000020:00200000:0.0:1789760798.061408:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.061416:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.061432:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b9c0 00000100:00100000:0.0:1789760798.061455:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613440:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.061675:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18608, 18608] 00000080:00200000:1.0:1789760798.061688:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.061693:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.061704:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76222464 i_blocks 148896 00000020:00200000:1.0:1789760798.061707:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.061715:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.061739:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76222464, count: 4096 00000080:00200000:1.0:1789760798.061746:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76222464, bytes: 4096 00000080:00200000:1.0:1789760798.061752:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.061760:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.061762:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76222464, 76226560) 0 0 00000080:00200000:1.0:1789760798.061765:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.061778:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18609, 18609] 00020000:00200000:1.0:1789760798.061784:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76222464 chunk: [76222464, 76226560) 76226560, 4096 00020000:00200000:1.0:1789760798.061788:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.061803:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18609, 18609] 00000008:00010000:1.0:1789760798.061816:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.061821:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.061841:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76222464, 76226560) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.061851:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76222464=0x48b1000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.061884:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.061887:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76226560 00000020:00010000:1.0:1789760798.061893:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.061896:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.061899:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.061902:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.061904:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.061910:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.061915:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760798.062038:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b240 00000008:00100000:0.0:1789760798.062131:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76222464<>76226560 00000100:00100000:0.0:1789760798.062177:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565613568:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.072043:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985917 gen 3 00000008:00010000:0.0:1789760798.072082:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b240 00000020:00200000:0.0:1789760798.072088:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.072097:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.072118:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b240 00000100:00100000:0.0:1789760798.072140:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565613568:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.072230:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18609, 18609] 00000080:00200000:1.0:1789760798.072244:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.072249:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.072261:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76226560 i_blocks 148904 00000020:00200000:1.0:1789760798.072264:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.072271:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.072308:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76226560, count: 4096 00000080:00200000:1.0:1789760798.072315:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76226560, bytes: 4096 00000080:00200000:1.0:1789760798.072323:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.072331:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.072333:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76226560, 76230656) 0 0 00000080:00200000:1.0:1789760798.072337:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.072350:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18610, 18610] 00020000:00200000:1.0:1789760798.072357:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76226560 chunk: [76226560, 76230656) 76230656, 4096 00020000:00200000:1.0:1789760798.072370:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.072386:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18610, 18610] 00000008:00010000:1.0:1789760798.072399:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.072404:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.072412:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76226560, 76230656) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.072422:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76226560=0x48b2000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.072466:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.072469:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76230656 00000020:00010000:1.0:1789760798.072475:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.072489:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.072493:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.072496:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.072499:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.072506:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.072511:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.072525:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b6c0 00000008:00100000:1.0:1789760798.072615:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb800 - object 0x240000400:5 offset 76226560<>76230656 00000100:00100000:0.0:1789760798.073064:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613696:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.092773:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985918 gen 3 00000008:00010000:0.0:1789760798.092811:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b6c0 00000020:00200000:0.0:1789760798.092818:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.092837:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.092859:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b6c0 00000100:00100000:0.0:1789760798.092883:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613696:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.093036:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18610, 18610] 00000080:00200000:1.0:1789760798.093049:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.093053:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.093065:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76230656 i_blocks 148912 00000020:00200000:1.0:1789760798.093068:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.093076:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.093111:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76230656, count: 4096 00000080:00200000:1.0:1789760798.093118:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76230656, bytes: 4096 00000080:00200000:1.0:1789760798.093129:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.093136:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.093138:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76230656, 76234752) 0 0 00000080:00200000:1.0:1789760798.093142:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.093154:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18611, 18611] 00020000:00200000:1.0:1789760798.093160:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76230656 chunk: [76230656, 76234752) 76234752, 4096 00020000:00200000:1.0:1789760798.093164:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.093180:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18611, 18611] 00000008:00010000:1.0:1789760798.093193:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.093198:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.093206:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76230656, 76234752) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.093217:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76230656=0x48b3000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.093265:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.093269:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76234752 00000020:00010000:1.0:1789760798.093275:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.093278:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.093281:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.093284:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.093287:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.093293:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.093298:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.093312:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bc00 00000008:00100000:1.0:1789760798.093418:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccb480 - object 0x240000400:5 offset 76230656<>76234752 00000100:00100000:0.0:1789760798.094023:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760798.094041:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613824:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.114112:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985919 gen 3 00000008:00010000:0.0:1789760798.114150:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bc00 00000020:00200000:0.0:1789760798.114157:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.114166:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.114183:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bc00 00000100:00100000:0.0:1789760798.114207:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613824:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.114815:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18611, 18611] 00000080:00200000:1.0:1789760798.114830:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.114835:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.114848:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76234752 i_blocks 148920 00000020:00200000:1.0:1789760798.114852:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.114860:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.114885:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76234752, count: 4096 00000080:00200000:1.0:1789760798.114891:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76234752, bytes: 4096 00000080:00200000:1.0:1789760798.114900:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.114907:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.114909:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76234752, 76238848) 0 0 00000080:00200000:1.0:1789760798.114913:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.114926:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18612, 18612] 00020000:00200000:1.0:1789760798.114933:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76234752 chunk: [76234752, 76238848) 76238848, 4096 00020000:00200000:1.0:1789760798.114937:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.114951:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18612, 18612] 00000008:00010000:1.0:1789760798.114964:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.114968:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.114991:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76234752, 76238848) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.115002:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76234752=0x48b4000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.115050:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.115054:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76238848 00000020:00010000:1.0:1789760798.115060:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.115064:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.115067:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.115070:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.115073:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.115079:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.115085:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.115214:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b540 00000008:00100000:1.0:1789760798.115310:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8e00 - object 0x240000400:5 offset 76234752<>76238848 00000100:00100000:0.0:1789760798.115385:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613952:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.152922:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985920 gen 3 00000008:00010000:0.0:1789760798.152959:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b540 00000020:00200000:0.0:1789760798.152966:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.152985:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.153001:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b540 00000100:00100000:0.0:1789760798.153025:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565613952:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.153120:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18612, 18612] 00000080:00200000:1.0:1789760798.153134:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.153139:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.153151:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76238848 i_blocks 148928 00000020:00200000:1.0:1789760798.153155:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.153162:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.153186:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76238848, count: 4096 00000080:00200000:1.0:1789760798.153193:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76238848, bytes: 4096 00000080:00200000:1.0:1789760798.153200:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.153206:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.153208:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76238848, 76242944) 0 0 00000080:00200000:1.0:1789760798.153212:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.153225:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18613, 18613] 00020000:00200000:1.0:1789760798.153231:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76238848 chunk: [76238848, 76242944) 76242944, 4096 00020000:00200000:1.0:1789760798.153236:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.153249:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18613, 18613] 00000008:00010000:1.0:1789760798.153261:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.153266:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.153274:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76238848, 76242944) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.153284:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76238848=0x48b5000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.153324:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.153327:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76242944 00000020:00010000:1.0:1789760798.153334:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.153337:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.153339:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.153342:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.153345:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.153350:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.153356:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.153380:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bf00 00000008:00100000:1.0:1789760798.153470:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8700 - object 0x240000400:5 offset 76238848<>76242944 00000100:00100000:0.0:1789760798.153616:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760798.153631:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565614080:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.186617:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985921 gen 3 00000008:00010000:0.0:1789760798.186683:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bf00 00000020:00200000:0.0:1789760798.186690:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.186697:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.186705:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bf00 00000100:00100000:0.0:1789760798.186727:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565614080:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.186817:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18613, 18613] 00000080:00200000:1.0:1789760798.186830:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.186835:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.186847:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76242944 i_blocks 148936 00000020:00200000:1.0:1789760798.186850:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.186858:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.186882:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76242944, count: 4096 00000080:00200000:1.0:1789760798.186887:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76242944, bytes: 4096 00000080:00200000:1.0:1789760798.186894:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.186901:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.186903:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76242944, 76247040) 0 0 00000080:00200000:1.0:1789760798.186907:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.186919:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18614, 18614] 00020000:00200000:1.0:1789760798.186926:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76242944 chunk: [76242944, 76247040) 76247040, 4096 00020000:00200000:1.0:1789760798.186929:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.186942:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18614, 18614] 00000008:00010000:1.0:1789760798.186956:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.186961:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.186970:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76242944, 76247040) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.186980:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76242944=0x48b6000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.187023:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.187026:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76247040 00000020:00010000:1.0:1789760798.187033:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.187036:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.187039:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.187042:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.187045:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.187050:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.187056:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.187069:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0bb40 00000008:00100000:1.0:1789760798.187158:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cca680 - object 0x240000400:5 offset 76242944<>76247040 00000100:00100000:0.0:1789760798.187284:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565614208:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.199966:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985922 gen 3 00000008:00010000:0.0:1789760798.200002:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0bb40 00000020:00200000:0.0:1789760798.200009:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.200017:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.200071:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0bb40 00000100:00100000:0.0:1789760798.200105:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565614208:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.200236:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18614, 18614] 00000080:00200000:1.0:1789760798.200249:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.200254:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.200265:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76247040 i_blocks 148944 00000020:00200000:1.0:1789760798.200269:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.200276:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.200301:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76247040, count: 4096 00000080:00200000:1.0:1789760798.200307:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76247040, bytes: 4096 00000080:00200000:1.0:1789760798.200315:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.200321:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.200324:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76247040, 76251136) 0 0 00000080:00200000:1.0:1789760798.200327:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.200340:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18615, 18615] 00020000:00200000:1.0:1789760798.200347:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76247040 chunk: [76247040, 76251136) 76251136, 4096 00020000:00200000:1.0:1789760798.200351:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.200461:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18615, 18615] 00000008:00010000:1.0:1789760798.200474:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.200740:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.200753:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76247040, 76251136) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.200764:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76247040=0x48b7000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.200810:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.200814:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76251136 00000020:00010000:1.0:1789760798.200821:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.200825:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.200828:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.200831:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.200834:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.200841:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.200846:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.200861:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b480 00000008:00100000:1.0:1789760798.200957:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8a80 - object 0x240000400:5 offset 76247040<>76251136 00000100:00100000:0.0:1789760798.201022:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565614336:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.217356:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985923 gen 3 00000008:00010000:0.0:1789760798.217402:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b480 00000020:00200000:0.0:1789760798.217409:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.217417:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.217435:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b480 00000100:00100000:0.0:1789760798.217457:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565614336:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.217558:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18615, 18615] 00000080:00200000:1.0:1789760798.217571:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.217576:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.217587:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76251136 i_blocks 148952 00000020:00200000:1.0:1789760798.217591:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.217598:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.217625:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76251136, count: 4096 00000080:00200000:1.0:1789760798.217632:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76251136, bytes: 4096 00000080:00200000:1.0:1789760798.217640:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.217687:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.217689:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76251136, 76255232) 0 0 00000080:00200000:1.0:1789760798.217693:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.217715:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18616, 18616] 00020000:00200000:1.0:1789760798.217723:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76251136 chunk: [76251136, 76255232) 76255232, 4096 00020000:00200000:1.0:1789760798.217726:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.217742:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18616, 18616] 00000008:00010000:1.0:1789760798.217756:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.217762:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.217771:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76251136, 76255232) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.217782:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76251136=0x48b8000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.217827:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.217830:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76255232 00000020:00010000:1.0:1789760798.217835:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.217839:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.217842:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.217845:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.217847:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.217852:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.217857:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.217870:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760798.217960:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc9500 - object 0x240000400:5 offset 76251136<>76255232 00000100:00100000:0.0:1789760798.218348:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565614464:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760798.234960:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760798.235392:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985924 gen 3 00000008:00010000:0.0:1789760798.235426:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760798.235432:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.235440:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.235458:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000080:00200000:1.0:1789760798.235577:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18616, 18616] 00000080:00200000:1.0:1789760798.235592:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.235597:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.235608:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76255232 i_blocks 148960 00000020:00200000:1.0:1789760798.235612:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.235620:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.235679:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76255232, count: 4096 00000080:00200000:1.0:1789760798.235687:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76255232, bytes: 4096 00000080:00200000:1.0:1789760798.235694:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.235702:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.235704:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76255232, 76259328) 0 0 00000080:00200000:1.0:1789760798.235717:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.235733:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18617, 18617] 00020000:00200000:1.0:1789760798.235740:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76255232 chunk: [76255232, 76259328) 76259328, 4096 00020000:00200000:1.0:1789760798.235744:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.235760:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18617, 18617] 00000008:00010000:1.0:1789760798.235773:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.235778:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.235786:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76255232, 76259328) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.235797:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76255232=0x48b9000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.235835:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.235837:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76259328 00000020:00010000:1.0:1789760798.235843:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.235846:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.235849:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.235852:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.235854:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.235860:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.235865:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:0.0:1789760798.235992:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565614464:192.168.203.147@tcp:4:dd.0 00000008:00010000:0.0:1789760798.236028:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:0.0:1789760798.236121:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76255232<>76259328 00000100:00100000:0.0:1789760798.236170:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565614592:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.250208:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985925 gen 3 00000008:00010000:0.0:1789760798.250243:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760798.250249:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.250258:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.250277:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760798.250301:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565614592:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.250397:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18617, 18617] 00000080:00200000:1.0:1789760798.250410:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.250415:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.250429:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76259328 i_blocks 148968 00000020:00200000:1.0:1789760798.250432:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.250439:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.250466:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76259328, count: 4096 00000080:00200000:1.0:1789760798.250473:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76259328, bytes: 4096 00000080:00200000:1.0:1789760798.250479:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.250497:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.250499:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76259328, 76263424) 0 0 00000080:00200000:1.0:1789760798.250503:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.250516:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18618, 18618] 00020000:00200000:1.0:1789760798.250531:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76259328 chunk: [76259328, 76263424) 76263424, 4096 00020000:00200000:1.0:1789760798.250535:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.250549:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18618, 18618] 00000008:00010000:1.0:1789760798.250564:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.250569:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.250578:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76259328, 76263424) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.250588:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76259328=0x48ba000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.250634:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.250637:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76263424 00000020:00010000:1.0:1789760798.250680:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.250684:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.250687:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.250690:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.250692:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.250699:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.250704:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.250718:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b0c0 00000008:00100000:1.0:1789760798.250832:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844cc8380 - object 0x240000400:5 offset 76259328<>76263424 00000100:00100000:0.0:1789760798.251249:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565614720:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.270456:0:2357:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985926 gen 3 00000008:00010000:0.0:1789760798.270494:0:2357:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b0c0 00000020:00200000:0.0:1789760798.270500:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.270508:0:2357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.270569:0:2357:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b0c0 00000100:00100000:0.0:1789760798.270594:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565614720:192.168.203.147@tcp:4:dd.0 00000080:00200000:1.0:1789760798.273076:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18618, 18618] 00000080:00200000:1.0:1789760798.273161:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1789760798.273167:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:1.0:1789760798.273200:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76263424 i_blocks 148976 00000020:00200000:1.0:1789760798.273204:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1789760798.273215:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:1.0:1789760798.273242:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76263424, count: 4096 00000080:00200000:1.0:1789760798.273250:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76263424, bytes: 4096 00000080:00200000:1.0:1789760798.273262:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:1.0:1789760798.273270:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:1.0:1789760798.273272:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76263424, 76267520) 0 0 00000080:00200000:1.0:1789760798.273276:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1789760798.273288:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18619, 18619] 00020000:00200000:1.0:1789760798.273295:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76263424 chunk: [76263424, 76267520) 76267520, 4096 00020000:00200000:1.0:1789760798.273375:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1789760798.273394:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18619, 18619] 00000008:00010000:1.0:1789760798.273409:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:1.0:1789760798.273414:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8588c0230, osc lock ffff93f852124850, flags 0x0 00000080:00200000:1.0:1789760798.273424:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76263424, 76267520) append 0 append lockpos 0 00000080:00200000:1.0:1789760798.273435:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76263424=0x48bb000, pages 1 (max 131072), lockless, parallel 00000020:00200000:1.0:1789760798.273494:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1789760798.273497:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76267520 00000020:00010000:1.0:1789760798.273503:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:1.0:1789760798.273507:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.273510:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8588c0230 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1789760798.273513:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:1.0:1789760798.273516:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8588c0230 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:1.0:1789760798.273522:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:1.0:1789760798.273528:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:1.0:1789760798.273541:0:53:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f843e0b780 00000008:00100000:1.0:1789760798.273639:0:53:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f844ccad80 - object 0x240000400:5 offset 76263424<>76267520 00000100:00100000:0.0:1789760798.275099:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565614848:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.294239:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985927 gen 3 00000008:00010000:0.0:1789760798.294280:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f843e0b780 00000020:00200000:0.0:1789760798.294286:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.294296:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.294325:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f843e0b780 00000100:00100000:0.0:1789760798.294353:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565614848:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760798.295507:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18619, 18619] 00000080:00200000:2.0:1789760798.295527:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760798.295533:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760798.295547:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76267520 i_blocks 148984 00000020:00200000:2.0:1789760798.295550:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760798.295561:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760798.295591:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76267520, count: 4096 00000080:00200000:2.0:1789760798.295598:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76267520, bytes: 4096 00000080:00200000:2.0:1789760798.295612:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760798.295620:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760798.295622:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76267520, 76271616) 0 0 00000080:00200000:2.0:1789760798.295626:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760798.295638:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18620, 18620] 00020000:00200000:2.0:1789760798.295683:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76267520 chunk: [76267520, 76271616) 76271616, 4096 00020000:00200000:2.0:1789760798.295688:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760798.295715:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18620, 18620] 00000008:00010000:2.0:1789760798.295730:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760798.295735:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760798.295745:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76267520, 76271616) append 0 append lockpos 0 00000080:00200000:2.0:1789760798.295756:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76267520=0x48bc000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760798.295805:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760798.295808:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76271616 00000020:00010000:2.0:1789760798.296153:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760798.296158:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760798.296163:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760798.296166:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760798.296169:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760798.296177:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760798.296183:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760798.296203:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760798.296300:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 76267520<>76271616 00000100:00100000:2.0:1789760798.296354:0:2359:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760798.296384:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565614976:192.168.203.147@tcp:4:dd.0 00000100:00100000:2.0:1789760798.313725:0:2359:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985928 gen 3 00000008:00010000:2.0:1789760798.313767:0:2359:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84354fe40 00000020:00200000:2.0:1789760798.313773:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:2.0:1789760798.313782:0:2359:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:2.0:1789760798.313802:0:2359:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84354fe40 00000100:00100000:2.0:1789760798.313828:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565614976:192.168.203.147@tcp:4:dd.0 00000080:00200000:0.0:1789760798.313927:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18620, 18620] 00000080:00200000:0.0:1789760798.313940:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1789760798.313946:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1789760798.313957:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76271616 i_blocks 148992 00000020:00200000:0.0:1789760798.313960:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1789760798.313968:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1789760798.313995:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76271616, count: 4096 00000080:00200000:0.0:1789760798.314001:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76271616, bytes: 4096 00000080:00200000:0.0:1789760798.314009:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:0.0:1789760798.314016:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:0.0:1789760798.314018:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76271616, 76275712) 0 0 00000080:00200000:0.0:1789760798.314022:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1789760798.314035:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18621, 18621] 00020000:00200000:0.0:1789760798.314041:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76271616 chunk: [76271616, 76275712) 76275712, 4096 00020000:00200000:0.0:1789760798.314045:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1789760798.314059:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18621, 18621] 00000008:00010000:0.0:1789760798.314074:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:0.0:1789760798.314078:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f850d7dbb0, osc lock ffff93f850af0260, flags 0x0 00000080:00200000:0.0:1789760798.314156:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76271616, 76275712) append 0 append lockpos 0 00000080:00200000:0.0:1789760798.314168:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76271616=0x48bd000, pages 1 (max 131072), lockless, parallel 00000020:00200000:0.0:1789760798.314217:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1789760798.314219:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76275712 00000020:00010000:0.0:1789760798.314226:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:0.0:1789760798.314229:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1789760798.314231:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f850d7dbb0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1789760798.314234:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:0.0:1789760798.314237:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f850d7dbb0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:0.0:1789760798.314244:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:0.0:1789760798.314249:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:0.0:1789760798.314263:0:448:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84874d180 00000008:00100000:0.0:1789760798.314356:0:448:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f850d41500 - object 0x240000400:5 offset 76271616<>76275712 00000100:00100000:0.0:1789760798.314406:0:2358:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760798.314419:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565615104:192.168.203.147@tcp:4:dd.0 00000100:00100000:0.0:1789760798.331223:0:2358:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff93f851b31000: committing for last_committed 4294985929 gen 3 00000008:00010000:0.0:1789760798.331259:0:2358:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff93f84874d180 00000020:00200000:0.0:1789760798.331266:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000020:00200000:0.0:1789760798.331274:0:2358:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1789760798.331299:0:2358:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff93f84874d180 00000100:00100000:0.0:1789760798.331323:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565615104:192.168.203.147@tcp:4:dd.0 00000080:00200000:2.0:1789760798.331431:0:11706:0:(file.c:2205:ll_file_io_generic()) Range unlock [18621, 18621] 00000080:00200000:2.0:1789760798.331443:0:11706:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1789760798.331448:0:11706:0:(file.c:2238:ll_file_io_generic()) tbf: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1789760798.331459:0:11706:0:(file.c:1775:ll_merge_attr_nolock()) [0x200000401:0x9:0x0] updating i_size 76275712 i_blocks 149000 00000020:00200000:2.0:1789760798.331462:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1789760798.331471:0:11706:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1789760798.331588:0:11706:0:(file.c:2650:do_file_write_iter()) START file tbf:[0x200000401:0x9:0x0], ppos: 76275712, count: 4096 00000080:00200000:2.0:1789760798.331596:0:11706:0:(file.c:2090:ll_file_io_generic()) tbf: write ppos: 76275712, bytes: 4096 00000080:00200000:2.0:1789760798.331605:0:11706:0:(file.c:1848:ll_io_set_mirror()) tbf: desiginated mirror: 0 00000020:00200000:2.0:1789760798.331613:0:11706:0:(cl_io.c:214:cl_io_rw_init()) header@00000000e5b19883[0x110, 2, [0x200000401:0x9:0x0] hash] 00000020:00200000:2.0:1789760798.331615:0:11706:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [76275712, 76279808) 0 0 00000080:00200000:2.0:1789760798.331619:0:11706:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1789760798.331632:0:11706:0:(file.c:2168:ll_file_io_generic()) Range lock [18622, 18622] 00020000:00200000:2.0:1789760798.331640:0:11706:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 76275712 chunk: [76275712, 76279808) 76279808, 4096 00020000:00200000:2.0:1789760798.331671:0:11706:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1789760798.331688:0:11706:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [18622, 18622] 00000008:00010000:2.0:1789760798.331700:0:11706:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 10 00000008:00010000:2.0:1789760798.331705:0:11706:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff93f8484a42b0, osc lock ffff93f845687560, flags 0x0 00000080:00200000:2.0:1789760798.331714:0:11706:0:(vvp_io.c:1292:vvp_io_write_start()) tbf: write [76275712, 76279808) append 0 append lockpos 0 00000080:00200000:2.0:1789760798.331725:0:11706:0:(rw26.c:362:ll_direct_IO()) VFS Op:inode=[0x200000401:0x9:0x0](ffff93f858d18908), size=4096 (max 536870912), offset=76275712=0x48be000, pages 1 (max 131072), lockless, parallel 00000020:00200000:2.0:1789760798.331766:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1789760798.331769:0:11706:0:(rw26.c:544:ll_direct_IO()) result 0 tot_bytes 4096 count 4096 file_offset 76279808 00000020:00010000:2.0:1789760798.331775:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_release():245 00000020:00010000:2.0:1789760798.331778:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_cancel():142 00000020:00010000:2.0:1789760798.331781:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff93f8484a42b0 (ffff93f84418aab8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1789760798.331784:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8507d03f8 (ffff93f84418a878/0) at cl_lock_fini():72 00000020:00010000:2.0:1789760798.331786:0:11706:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff93f8484a42b0 (ffff93f84418a878/1) at cl_lock_fini():72 00000080:00200000:2.0:1789760798.331792:0:11706:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000020:00200000:2.0:1789760798.331799:0:11706:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000008:00010000:2.0:1789760798.331813:0:11738:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff93f84354fe40 00000008:00100000:2.0:1789760798.331919:0:11738:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff93f8517c0700 - object 0x240000400:5 offset 76275712<>76279808 00000100:00100000:3.0:1789760798.332537:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565615232:192.168.203.147@tcp:4:dd.0 00010000:00010000:3.0:1789760799.260535:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760800.283903:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760801.307695:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760802.266832:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:af634c28-3647-4f73-bb24-40d7878858fb:2360:1876699565615360:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760802.266857:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565615488:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1789760802.266934:0:2360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565615616:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760802.266935:0:2359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2359:1876699565615744:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1789760802.330819:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760803.354798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1789760803.802764:0:8:0:(vvp_io.c:1826:vvp_io_init()) [0x200000401:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1789760803.802781:0:8:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1789760803.802804:0:8:0:(vvp_io.c:304:vvp_io_fini()) [0x200000401:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00010000:00010000:3.0:1789760804.378810:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760805.402793:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760806.426797:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760807.386811:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565616000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760807.386827:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:af634c28-3647-4f73-bb24-40d7878858fb:2358:1876699565615872:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1789760807.386895:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2357:1876699565616256:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760807.386902:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565616128:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1789760807.450797:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760808.474777:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760809.498785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760810.522818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760811.546744:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760812.506808:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:af634c28-3647-4f73-bb24-40d7878858fb:2358:1876699565616384:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760812.506816:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565616512:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1789760812.506898:0:2358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565616640:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760812.506898:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2357:1876699565616768:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1789760812.570817:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760813.594816:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:2.0:1789760814.554800:0:2359:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760798/real 1789760798] req@ffff93f8517c0700 x1876699565615232/t0(0) o4->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:6/4 lens 4584/448 e 0 to 1 dl 1789760814 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00080000:2.0:1789760814.564764:0:2359:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1789760814.564772:0:2359:0:(import.c:182:ptlrpc_set_import_discon()) lustre-OST0000-osc-ffff93f851b31000: Connection to lustre-OST0000 (at 192.168.203.147@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1789760814.570869:0:2359:0:(import.c:418:ptlrpc_pinger_force()) lustre-OST0000_UUID: waking up pinger s:DISCONN 00000100:00080000:1.0:1789760814.570981:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760814.570985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789760814.570988:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789760814.570998:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 394 00000100:00080000:1.0:1789760814.571009:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789760814.571018:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789760814.571031:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565617152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789760814.571074:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565617024:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760814.571074:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565617152:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789760814.571179:0:2357:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760814.571190:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:af634c28-3647-4f73-bb24-40d7878858fb:2357:1876699565616896:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760814.571230:0:2357:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2357:1876699565617280:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1789760814.618888:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760815.642837:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760816.666830:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760817.690807:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:2.0:1789760818.650776:0:2359:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760802/real 1789760802] req@ffff93f8517c0a80 x1876699565615488/t0(0) o400->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760818 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000400:3.0:1789760818.650776:0:2360:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760802/real 1789760802] req@ffff93f8517c3100 x1876699565615360/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1789760818 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760818.650832:0:2360:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f8517c3100 x1876699565615360/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1789760818 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760818.650842:0:2360:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from FULL to DISCONN 00000100:02020000:3.0:1789760818.650845:0:2360:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.147@tcp: Connection to MGS (at 192.168.203.147@tcp) was lost; in progress operations using this service will fail 00000100:00100000:2.0:1789760818.660411:0:2359:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f8517c0a80 x1876699565615488/t0(0) o400->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760818 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760818.660420:0:2359:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1789760818.660423:0:2359:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff93f851b31000: Connection to lustre-MDT0000 (at 192.168.203.147@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1789760818.660430:0:2359:0:(import.c:418:ptlrpc_pinger_force()) lustre-MDT0000_UUID: waking up pinger s:DISCONN 00000100:00000400:2.0:1789760818.660452:0:2359:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760802/real 1789760802] req@ffff93f8517c1180 x1876699565615744/t0(0) o400->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760818 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760818.660469:0:2359:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f8517c1180 x1876699565615744/t0(0) o400->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760818 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760818.660474:0:2359:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1789760818.660481:0:2359:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff93f850b8d000: Connection to lustre-MDT0000 (at 192.168.203.147@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1789760818.660484:0:2359:0:(import.c:418:ptlrpc_pinger_force()) lustre-MDT0000_UUID: waking up pinger s:DISCONN 00000100:00100000:2.0:1789760818.660515:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2359:1876699565615488:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760818.660520:0:2359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2359:1876699565615744:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1789760818.670432:0:2360:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.147@tcp for MGC192.168.203.147@tcp not replayable, auto-deactivating 00000100:00080000:3.0:1789760818.670436:0:2360:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:3.0:1789760818.670441:0:2360:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff93f8517c3100 x1876699565615360/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1789760818 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760818.670453:0:2360:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff93f844ccad80 x1876699565615872/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1789760823 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760818.670468:0:2360:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff93f844cca680 x1876699565616384/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1789760828 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760818.670475:0:2360:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff93f844ccb800 x1876699565616896/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1789760830 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:3.0:1789760818.670499:0:2360:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:3.0:1789760818.670502:0:2360:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:3.0:1789760818.670504:0:2360:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:3.0:1789760818.670506:0:2360:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1789760818.670530:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760818.670538:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789760818.670544:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789760818.670553:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 254 00000100:00100000:0.0:1789760818.670555:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:af634c28-3647-4f73-bb24-40d7878858fb:2358:1876699565616384:192.168.203.147@tcp:400:kworker.0 00000100:00080000:1.0:1789760818.670575:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00000400:2.0:1789760818.670577:0:2360:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760802/real 1789760802] req@ffff93f8517c2680 x1876699565615616/t0(0) o400->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 1 dl 1789760818 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760818.670595:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:af634c28-3647-4f73-bb24-40d7878858fb:2358:1876699565615872:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760818.670607:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:af634c28-3647-4f73-bb24-40d7878858fb:2357:1876699565616896:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760818.670620:0:2360:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f8517c2680 x1876699565615616/t0(0) o400->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 1 dl 1789760818 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760818.670635:0:2360:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff93f850c41800 already not connected (conn 4, was 3): CONNECTING 00000100:00100000:2.0:1789760818.670669:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:af634c28-3647-4f73-bb24-40d7878858fb:2360:1876699565615360:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1789760818.670676:0:2360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2360:1876699565615616:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760818.670718:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565617408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789760818.670743:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760818.670746:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789760818.670748:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789760818.670751:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 255 00000100:00080000:1.0:1789760818.670758:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789760818.670767:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789760818.670774:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565617408:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789760818.670782:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565617536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789760818.670792:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760818.670795:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1789760818.670798:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760818.670801:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789760818.670803:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789760818.670806:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 317 00000100:00080000:1.0:1789760818.670811:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789760818.670816:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789760818.670824:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565617664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789760818.670838:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760818.670841:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1789760818.670890:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565617536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760818.670945:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565617664:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760818.714816:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789760819.674764:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760819.674775:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1789760819.674778:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760819.674781:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1789760819.674783:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760819.674785:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1789760819.674803:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760819.674805:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760819.738854:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760820.762796:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760821.786814:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760822.810819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1789760823.642734:0:2358:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760807/real 1789760807] req@ffff93f844cc9500 x1876699565616128/t0(0) o400->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 1 dl 1789760823 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760823.652348:0:2358:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f844cc9500 x1876699565616128/t0(0) o400->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 1 dl 1789760823 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760823.652358:0:2358:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff93f850c41800 already not connected (conn 4, was 3): CONNECTING 00000100:00100000:0.0:1789760823.652391:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565616128:192.168.203.147@tcp:400:kworker.0 00000100:00000400:1.0:1789760823.706746:0:2357:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760807/real 1789760807] req@ffff93f844cc8380 x1876699565616000/t0(0) o400->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760823 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760823.706750:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760823.706757:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760823.706763:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760823.706767:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760823.706769:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760823.706771:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760823.706774:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760823.706776:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1789760823.706786:0:2357:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f844cc8380 x1876699565616000/t0(0) o400->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760823 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760823.706793:0:2357:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff93f850c45000 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:1.0:1789760823.706799:0:2357:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760807/real 1789760807] req@ffff93f844cc8a80 x1876699565616256/t0(0) o400->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760823 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760823.706807:0:2357:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f844cc8a80 x1876699565616256/t0(0) o400->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760823 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760823.706811:0:2357:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff93f850b8a800 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:1.0:1789760823.706824:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565616000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760823.706828:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2357:1876699565616256:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1789760823.834847:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760824.794726:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760814/real 1789760814] req@ffff93f844ccbb80 x1876699565617152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760824 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760824.794769:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565617152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760824 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760824.794787:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760824.794810:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760824.794814:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565617152:192.168.203.147@tcp:8:kworker.0 00000100:00080000:3.0:1789760824.794854:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760824.794860:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760824.794863:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760824.794940:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760824.794943:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760824.794946:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760824.794950:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760824.794957:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 754 00000100:00080000:3.0:1789760824.794966:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1789760824.794974:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760824.795004:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760824.795033:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565617792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760824.795056:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760824.795059:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1789760824.795089:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565617792:192.168.203.147@tcp:8:kworker.0 00010000:00010000:3.0:1789760824.858781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760825.882817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760826.906779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760827.931013:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1789760828.762760:0:2358:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760812/real 1789760812] req@ffff93f844cc8e00 x1876699565616640/t0(0) o400->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760828.773448:0:2358:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f844cc8e00 x1876699565616640/t0(0) o400->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760828.773474:0:2358:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff93f850c41800 already not connected (conn 5, was 3): CONNECTING 00000100:00100000:0.0:1789760828.773489:0:2358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2358:1876699565616640:192.168.203.147@tcp:400:kworker.0 00000100:00000400:1.0:1789760828.826792:0:2357:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760812/real 1789760812] req@ffff93f844cc8700 x1876699565616512/t0(0) o400->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760828.826840:0:2357:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f844cc8700 x1876699565616512/t0(0) o400->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760828.826850:0:2357:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff93f850c45000 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:1.0:1789760828.826857:0:2357:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760812/real 1789760812] req@ffff93f844ccb480 x1876699565616768/t0(0) o400->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760828.826870:0:2357:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f844ccb480 x1876699565616768/t0(0) o400->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760828.826876:0:2357:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff93f850b8a800 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:1.0:1789760828.826893:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565616512:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760828.826899:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2357:1876699565616768:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760828.890811:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760818/real 1789760818] req@ffff93f844ccaa00 x1876699565617536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760828.890836:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565617536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760828.890844:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760818/real 1789760818] req@ffff93f844cc8000 x1876699565617664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760828.890852:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565617664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760828.890858:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760818/real 1789760818] req@ffff93f844ccb800 x1876699565617408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760828.890865:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565617408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760828 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760828.890880:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760828.890901:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760828.890905:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565617536:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1789760828.890910:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760828.890914:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760828.890916:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565617664:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1789760828.890919:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760828.890923:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760828.890926:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565617408:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789760828.890944:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760828.890950:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789760828.890952:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760828.890959:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 758 00000100:00080000:0.0:1789760828.890968:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1789760828.890975:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789760828.891003:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565617920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760828.891022:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760828.891024:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760828.891026:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760828.891028:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 758 00000100:00080000:0.0:1789760828.891031:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1789760828.891034:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760828.891042:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789760828.891052:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565617920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789760828.891053:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565618048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760828.891063:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760828.891066:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760828.891068:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760828.891070:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760828.891071:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760828.891074:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 758 00000100:00080000:0.0:1789760828.891077:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1789760828.891080:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760828.891083:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760828.891091:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565618176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789760828.891153:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565618048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760828.891208:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565618176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760828.954817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789760829.914737:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760829.914746:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760829.914749:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760829.914753:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760829.914755:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760829.914758:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760829.914761:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760829.914764:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760829.978927:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:1.0:1789760830.874755:0:2357:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760814/real 1789760814] req@ffff93f844ccb100 x1876699565617280/t0(0) o400->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760830 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760830.874783:0:2357:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f844ccb100 x1876699565617280/t0(0) o400->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760830 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760830.874791:0:2357:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff93f850b8a800 already not connected (conn 3, was 1): CONNECTING 00000100:00000400:1.0:1789760830.874805:0:2357:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760814/real 1789760814] req@ffff93f844cc9c00 x1876699565617024/t0(0) o400->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760830 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760830.874818:0:2357:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff93f844cc9c00 x1876699565617024/t0(0) o400->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1789760830 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760830.874823:0:2357:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff93f850c45000 already not connected (conn 3, was 1): CONNECTING 00000100:00100000:1.0:1789760830.874836:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6a301392-4ca2-4c43-8653-61d4c47997a9:2357:1876699565617280:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1789760830.874842:0:2357:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2357:1876699565617024:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1789760831.002773:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760832.026819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789760832.922817:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760832.922829:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760832.922845:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760832.922850:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760832.922854:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760832.922858:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760832.922862:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760832.922866:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760833.050998:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760834.074833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760835.098820:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760836.122790:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760837.146822:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760838.106811:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760838.106824:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760838.106830:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760838.106838:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760838.106846:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760838.106852:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760838.106857:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760838.106864:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760838.170786:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760839.130837:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760824/real 1789760824] req@ffff93f844d65180 x1876699565617792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760839 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760839.130903:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565617792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760839 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760839.130955:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760839.130980:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760839.131000:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565617792:192.168.203.147@tcp:8:kworker.0 00000100:00080000:3.0:1789760839.131230:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760839.131242:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760839.131246:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760839.131251:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760839.131254:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760839.131258:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760839.131263:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760839.131275:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 764 00000100:00080000:3.0:1789760839.131299:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 10s 00000100:00080000:3.0:1789760839.131309:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760839.131340:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760839.131387:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565618304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760839.131430:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760839.131434:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1789760839.131490:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565618304:192.168.203.147@tcp:8:kworker.0 00010000:00010000:3.0:1789760839.194825:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760840.218759:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760841.242779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760842.266821:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760843.226848:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760828/real 1789760828] req@ffff93f850d41f80 x1876699565618048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760843 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760843.226892:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565618048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760843 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760843.226906:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760828/real 1789760828] req@ffff93f850d43100 x1876699565618176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760843 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760843.226921:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565618176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760843 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760843.226933:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760828/real 1789760828] req@ffff93f850d41500 x1876699565617920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760843 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760843.226948:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565617920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760843 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760843.226968:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760843.226990:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760843.226996:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565618048:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1789760843.227004:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760843.227013:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760843.227019:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760843.227022:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760843.227037:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789760843.227041:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760843.227042:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:2.0:1789760843.227056:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 768 00000100:00100000:3.0:1789760843.227057:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565618176:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1789760843.227070:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760843.227072:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 10s 00000100:00080000:3.0:1789760843.227082:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:2.0:1789760843.227084:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789760843.227091:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565617920:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789760843.227098:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789760843.227132:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565618432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789760843.227154:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760843.227156:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760843.227159:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760843.227161:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789760843.227162:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760843.227165:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 768 00000100:00080000:2.0:1789760843.227168:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1789760843.227171:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789760843.227175:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789760843.227182:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565618560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789760843.227195:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1789760843.227197:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565618432:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1789760843.227198:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789760843.227201:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760843.227209:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 768 00000100:00080000:2.0:1789760843.227212:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1789760843.227215:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789760843.227225:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565618688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789760843.227232:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760843.227234:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1789760843.227329:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565618560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789760843.227389:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565618688:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789760843.290893:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789760844.250706:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760844.250714:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760844.250717:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760844.250720:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760844.250723:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760844.250726:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760844.250728:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760844.250730:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760844.314765:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760845.338835:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760846.362860:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760847.386851:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789760848.282768:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760848.282777:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760848.282779:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760848.282782:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760848.282784:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760848.282787:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760848.282789:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760848.282791:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760848.410804:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760849.434891:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760850.458837:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760851.482825:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760852.506883:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789760853.466793:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760853.466803:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760853.466807:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760853.466812:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760853.466818:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760853.466821:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760853.466825:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760853.466829:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760853.530826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760854.554859:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760855.578857:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760856.602823:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760857.626862:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760858.586784:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760858.586794:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760858.586798:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760858.586802:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760858.586806:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760858.586809:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760858.586812:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760858.586814:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760858.650844:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760859.610729:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760839/real 1789760839] req@ffff93f844d65180 x1876699565618304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760859 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760859.610755:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565618304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760859 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760859.610782:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760859.610801:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760859.610805:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565618304:192.168.203.147@tcp:8:kworker.0 00000100:00080000:3.0:1789760859.610863:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760859.610869:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760859.610872:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760859.610874:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760859.610876:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760859.610878:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760859.610880:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760859.610886:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 779 00000100:00080000:3.0:1789760859.610894:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 15s 00000100:00080000:3.0:1789760859.610900:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760859.610937:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760859.610964:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565618816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760859.610983:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760859.610985:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1789760859.611013:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565618816:192.168.203.147@tcp:8:kworker.0 00010000:00010000:3.0:1789760859.674886:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760860.698862:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760861.722931:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760862.746904:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:1.0:1789760863.066756:0:2348:0:(socklnd_cb.c:2254:ksocknal_find_timed_out_conn()) A connection with 12345-192.168.203.147@tcp (192.168.203.147:988) timed out; the network or node may be down. 00000100:00100000:1.0:1789760863.706715:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760843/real 1789760843] req@ffff93f8517c3100 x1876699565618560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760863 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760863.706751:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565618560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760863 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760863.706761:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760843/real 1789760843] req@ffff93f8517c1180 x1876699565618688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760863 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760863.706772:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565618688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760863 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760863.706780:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760843/real 1789760843] req@ffff93f8517c2680 x1876699565618432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760863 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760863.706791:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565618432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760863 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760863.706808:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760863.706824:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760863.706828:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565618560:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1789760863.706832:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760863.706838:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760863.706843:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565618688:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789760863.706849:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760863.706854:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760863.706860:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789760863.706862:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:0.0:1789760863.706865:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789760863.706868:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565618432:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1789760863.706891:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 783 00000100:00080000:0.0:1789760863.706902:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 15s 00000100:00080000:0.0:1789760863.706908:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789760863.706950:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565618944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760863.706968:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760863.706971:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760863.706973:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760863.706976:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 783 00000100:00080000:0.0:1789760863.706979:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 15s 00000100:00080000:0.0:1789760863.706982:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760863.706991:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789760863.707000:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565618944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789760863.707002:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565619072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760863.707016:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760863.707021:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1789760863.707023:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760863.707025:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760863.707026:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760863.707029:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 783 00000100:00080000:0.0:1789760863.707032:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 15s 00000100:00080000:0.0:1789760863.707034:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760863.707037:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760863.707044:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565619200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760863.707055:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760863.707057:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1789760863.707097:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565619072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760863.707122:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565619200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760863.770838:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789760864.730847:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760864.730861:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1789760864.730883:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760864.730888:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1789760864.730893:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760864.730897:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1789760864.730903:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760864.730907:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760864.794787:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760865.818896:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760866.842877:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760867.866908:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789760868.762783:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760868.762795:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760868.762799:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760868.762802:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760868.762804:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760868.762806:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760868.762808:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760868.762811:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760868.890909:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760869.914870:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760870.938932:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760871.962940:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760872.986843:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789760873.946836:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760873.946848:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760873.946868:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760873.946871:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760873.946873:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760873.946875:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760873.946878:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760873.946880:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760874.010882:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760875.034829:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760876.058795:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760877.082766:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760878.106770:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789760879.066750:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760879.066788:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760879.066791:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760879.066795:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760879.066798:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760879.066800:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760879.066803:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760879.066806:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760879.130815:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760880.154780:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760881.178886:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760882.202775:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760883.226780:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760884.186790:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789760859/real 1789760859] req@ffff93f844d66300 x1876699565618816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760884 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760884.186792:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1789760884.186853:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565618816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760884 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760884.186880:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760884.186891:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760884.186896:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760884.186900:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760884.186903:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760884.186908:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760884.186913:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789760884.186918:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1789760884.186928:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 799 00000100:00100000:1.0:1789760884.186931:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565618816:192.168.203.147@tcp:8:kworker.0 00000100:00080000:3.0:1789760884.186942:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760884.186953:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760884.186967:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760884.187004:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565619328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760884.187029:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760884.187031:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760884.187039:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760884.187043:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760884.187045:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760884.187047:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760884.187049:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760884.187052:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1789760884.187072:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565619328:192.168.203.147@tcp:8:kworker.0 00010000:00010000:3.0:1789760884.250874:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760885.274778:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760886.298776:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760887.322777:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760888.282762:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1789760863/real 0] req@ffff93f850d40000 x1876699565619072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760888 ref 2 fl Rpc:XNr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760888.282792:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565619072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760888 ref 2 fl UnregRPC:XNU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760888.282801:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1789760863/real 0] req@ffff93f850d42680 x1876699565619200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760888 ref 2 fl Rpc:XNr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760888.282809:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565619200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760888 ref 2 fl UnregRPC:XNU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760888.282816:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1789760863/real 0] req@ffff93f850d40700 x1876699565618944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760888 ref 2 fl Rpc:XNr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760888.282824:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565618944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760888 ref 2 fl UnregRPC:XNU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1789760888.346805:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760889.306745:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760889.306753:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760889.306756:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760889.306759:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760889.306761:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760889.306763:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1789760889.306765:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760889.306769:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760889.370834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760890.394811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760891.418797:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760892.442854:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760893.466766:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789760894.427073:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760894.427086:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760894.427092:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760894.427095:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760894.427098:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760894.427101:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1789760894.427104:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760894.427106:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1789760894.490856:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760895.514793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760896.538809:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760897.562770:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789760898.522840:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.147:988 00000400:00000100:3.0:1789760898.522885:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789760898.522900:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789760898.522937:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789760898.522948:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789760898.522956:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789760898.523000:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760884/real 1789760898] req@ffff93f844d64a80 x1876699565619328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760898.523021:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565619328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760898.523030:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760898.523048:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760898.523051:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565619328:192.168.203.147@tcp:8:kworker.0 00000100:00080000:1.0:1789760898.523056:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760898.523060:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760898.523063:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565619072:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1789760898.523070:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760898.523074:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760898.523081:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1789760898.523085:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1789760898.523086:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760898.523093:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:1.0:1789760898.523094:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565619200:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1789760898.523098:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789760898.523103:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760898.523108:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 803 00000100:00080000:1.0:1789760898.523114:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1789760898.523116:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760898.523125:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789760898.523132:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565618944:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789760898.523140:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760898.523170:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565619456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760898.523198:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760898.523202:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760898.523204:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760898.523209:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 824 00000100:00080000:3.0:1789760898.523214:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760898.523219:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760898.523225:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789760898.523226:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565619456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789760898.523241:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565619584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760898.523255:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760898.523259:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760898.523262:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760898.523266:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 803 00000100:00080000:3.0:1789760898.523270:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760898.523274:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760898.523280:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789760898.523290:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760898.523296:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565619712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760898.523314:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760898.523317:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789760898.523319:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789760898.523325:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760898/real 1789760898] req@ffff93f844d66d80 x1876699565619456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760898.523325:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 803 00000100:00080000:3.0:1789760898.523334:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760898.523341:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789760898.523344:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565619456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760898.523351:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760898.523356:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760898.523358:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565619456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789760898.523370:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565619840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789760898.523373:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565619584:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789760898.523398:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789760898.523408:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760898/real 1789760898] req@ffff93f844d65f80 x1876699565619584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760898.523415:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565619584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760898.523425:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760898.523427:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760898.523429:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565619584:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789760898.523433:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565619712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789760898.523443:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789760898.523448:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760898/real 1789760898] req@ffff93f844d64380 x1876699565619712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760898.523455:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565619712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760898.523460:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760898.523462:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760898.523464:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565619712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760898.523473:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565619840:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1789760898.523483:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789760898.523492:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760898/real 1789760898] req@ffff93f844d65500 x1876699565619840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760898.523499:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565619840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760898.523504:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760898.523506:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760898.523508:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565619840:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789760898.586780:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760899.610768:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760900.634790:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760901.658790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760902.682764:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760903.642847:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760903.642868:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789760903.642877:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760903.642896:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 838 00000100:00080000:3.0:1789760903.642916:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760903.642932:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789760903.643006:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565619968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760903.643043:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760903.643049:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760903.643052:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760903.643058:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 838 00000100:00080000:3.0:1789760903.643064:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760903.643071:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760903.643082:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789760903.643095:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565619968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789760903.643098:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565620096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760903.643123:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760903.643127:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760903.643129:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760903.643134:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 838 00000100:00080000:3.0:1789760903.643138:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760903.643141:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760903.643147:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760903.643157:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565620224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760903.643166:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760903.643170:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760903.643171:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760903.643175:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 838 00000100:00080000:3.0:1789760903.643178:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760903.643181:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760903.643185:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760903.643197:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565620352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789760903.643225:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565620096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760903.643256:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565620224:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789760903.643271:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565620352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760903.706808:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760904.730836:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760905.754846:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789760906.714974:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.147:988 00000400:00000100:1.0:1789760906.715021:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789760906.715038:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789760906.715173:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760903/real 1789760906] req@ffff93f844d64e00 x1876699565619968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760928 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760906.715202:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565619968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760928 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760906.715216:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760906.715221:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760906.715227:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565619968:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1789760906.715607:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789760906.715634:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789760906.715689:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760903/real 1789760906] req@ffff93f844d64000 x1876699565620096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000100:1.0:1789760906.715690:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789760906.715715:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565620096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760906.715727:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760906.715732:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760906.715736:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565620096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789760906.715745:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760903/real 1789760906] req@ffff93f844d65c00 x1876699565620224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760906.715762:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565620224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760906.715773:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760906.715779:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760906.715785:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565620224:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789760906.715795:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760903/real 1789760906] req@ffff93f844d66a00 x1876699565620352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760906.715811:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565620352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760906.715821:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760906.715825:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760906.715829:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565620352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760906.778877:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760907.802779:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789760908.762991:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760908.763011:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789760908.763020:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760908.763038:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 843 00000100:00080000:0.0:1789760908.763062:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760908.763077:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789760908.763157:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565620480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760908.763208:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760908.763215:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760908.763219:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760908.763229:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 843 00000100:00080000:0.0:1789760908.763237:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760908.763246:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760908.763262:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789760908.763280:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565620480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789760908.763291:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565620608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760908.763315:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760908.763322:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789760908.763325:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760908.763333:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 843 00000100:00080000:0.0:1789760908.763340:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760908.763348:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760908.763358:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789760908.763383:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565620736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760908.763402:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760908.763409:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760908.763412:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760908.763421:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 843 00000100:00080000:0.0:1789760908.763428:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760908.763435:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789760908.763442:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565620608:192.168.203.147@tcp:38:kworker.0 00000008:00100000:0.0:1789760908.763449:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760908.763478:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565620864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789760908.763498:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565620736:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789760908.763615:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565620864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760908.826814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789760909.786955:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789760909.786999:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789760909.787016:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789760909.787053:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789760909.787064:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789760909.787074:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789760909.787107:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760908/real 1789760909] req@ffff93f850d41c00 x1876699565620864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760909.787134:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565620864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760909.787146:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760909.787152:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760909.787157:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565620864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760909.787166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760908/real 1789760909] req@ffff93f850d40e00 x1876699565620608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760909.787178:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565620608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760909.787186:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760909.787189:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760909.787192:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565620608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760909.787198:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760908/real 1789760909] req@ffff93f850d40380 x1876699565620736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760909.787209:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565620736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760909.787217:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760909.787221:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760909.787224:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565620736:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789760909.787230:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760908/real 1789760909] req@ffff93f850d43b80 x1876699565620480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760909.787241:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565620480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760909.787248:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760909.787252:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760909.787255:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565620480:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789760909.850831:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760910.874763:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760911.898819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760912.922773:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789760913.882726:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760913.882735:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789760913.882738:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760913.882747:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 848 00000100:00080000:0.0:1789760913.882755:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760913.882762:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789760913.882801:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565620992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760913.882818:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760913.882821:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760913.882823:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760913.882826:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 848 00000100:00080000:0.0:1789760913.882828:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760913.882831:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760913.882838:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760913.882846:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42a00 x1876699565621120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760913.882854:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760913.882857:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789760913.882858:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760913.882861:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 848 00000100:00080000:0.0:1789760913.882862:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760913.882864:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760913.882868:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789760913.882873:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565620992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789760913.882879:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565621248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760913.882888:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760913.882890:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760913.882891:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760913.882894:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 848 00000100:00080000:0.0:1789760913.882896:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760913.882899:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760913.882901:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760913.882910:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565621376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789760913.882983:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565621120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760913.883029:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565621248:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789760913.883057:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565621376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760913.946834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760914.970825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760915.994787:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789760916.954945:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789760916.954989:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789760916.955006:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789760916.955051:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789760916.955060:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789760916.955066:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789760916.955134:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760913/real 1789760916] req@ffff93f850d43b80 x1876699565621248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760916.955166:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565621248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760916.955180:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760916.955186:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760916.955190:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565621248:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789760916.955200:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760913/real 1789760916] req@ffff93f850d40380 x1876699565621376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760916.955214:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565621376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760916.955223:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760916.955226:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760916.955230:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565621376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760916.955238:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760913/real 1789760916] req@ffff93f850d42300 x1876699565620992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760916.955251:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565620992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760916.955259:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760916.955263:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760916.955267:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565620992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789760916.955274:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760913/real 1789760916] req@ffff93f850d42a00 x1876699565621120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760916.955286:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42a00 x1876699565621120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760916.955294:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760916.955297:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760916.955301:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565621120:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760917.018810:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760918.042804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789760919.002759:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760919.002767:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789760919.002771:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760919.002780:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 853 00000100:00080000:2.0:1789760919.002790:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760919.002800:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789760919.002843:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565621504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789760919.002881:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760919.002885:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789760919.002888:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760919.002893:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 853 00000100:00080000:2.0:1789760919.002901:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760919.002907:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789760919.002921:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760919.002934:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565621504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789760919.002958:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565621632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789760919.002977:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760919.002983:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789760919.002986:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760919.002991:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 853 00000100:00080000:2.0:1789760919.002996:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760919.003002:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789760919.003009:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789760919.003026:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565621760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789760919.003039:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760919.003044:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:3.0:1789760919.003052:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789760919.003053:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760919.003065:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 853 00000100:00080000:2.0:1789760919.003074:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760919.003081:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789760919.003088:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789760919.003107:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565621888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789760919.003144:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760919/real 1789760919] req@ffff93f8517c0a80 x1876699565621504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760919.003174:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565621504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760919.003190:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760919.003196:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760919.003201:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565621504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789760919.003244:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565621632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789760919.003278:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760919.003295:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760919/real 1789760919] req@ffff93f8517c1500 x1876699565621632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760919.003308:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565621632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760919.003318:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760919.003322:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760919.003325:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565621632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789760919.003339:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565621760:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789760919.003364:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760919.003377:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760919/real 1789760919] req@ffff93f8517c0380 x1876699565621760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760919.003388:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565621760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760919.003397:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760919.003400:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760919.003403:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565621760:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789760919.003410:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565621888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789760919.003432:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760919.003444:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760919/real 1789760919] req@ffff93f8517c2300 x1876699565621888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760919.003478:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565621888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760919.003486:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760919.003489:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760919.003493:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565621888:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760919.066783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760920.090880:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760921.114822:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760922.138836:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760923.162831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789760924.122751:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760924.122763:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789760924.122767:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760924.122778:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 859 00000100:00080000:2.0:1789760924.122788:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760924.122796:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789760924.122836:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565622016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789760924.122870:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760924.122876:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789760924.122880:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760924.122886:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 859 00000100:00080000:2.0:1789760924.122893:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760924.122901:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789760924.122914:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760924.122918:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565622016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789760924.122937:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565622144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789760924.122951:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760924.122955:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789760924.122968:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760924.122973:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 859 00000100:00080000:2.0:1789760924.122978:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760924.122982:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789760924.122988:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789760924.123003:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565622272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789760924.123014:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1789760924.123017:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789760924.123020:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789760924.123025:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760924.123033:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 859 00000100:00080000:2.0:1789760924.123038:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760924.123044:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789760924.123049:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789760924.123062:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565622400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789760924.123074:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760924/real 1789760924] req@ffff93f8517c0000 x1876699565622016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760924.123099:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565622016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760924.123111:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760924.123116:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760924.123120:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565622016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789760924.123144:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565622144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789760924.123196:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760924.123218:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760924/real 1789760924] req@ffff93f8517c3b80 x1876699565622144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760924.123238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565622144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760924.123247:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760924.123250:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760924.123254:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565622144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789760924.123260:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565622272:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789760924.123286:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760924.123303:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760924/real 1789760924] req@ffff93f8517c1f80 x1876699565622272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760924.123331:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565622272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760924.123343:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760924.123348:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760924.123352:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565622272:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789760924.123360:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565622400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789760924.123401:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760924.123438:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760924/real 1789760924] req@ffff93f8517c2300 x1876699565622400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760924.123465:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565622400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760924.123487:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760924.123492:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760924.123498:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565622400:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760924.186820:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760925.210807:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760926.234817:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760927.258856:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760928.282779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760929.242746:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760929.242760:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789760929.242766:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760929.242780:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 864 00000100:00080000:3.0:1789760929.242793:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760929.242804:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789760929.242854:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565622528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760929.242902:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760929.242910:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760929.242912:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760929.242918:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 864 00000100:00080000:3.0:1789760929.242922:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760929.242927:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760929.242938:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760929.242953:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565622656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789760929.242962:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565622528:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789760929.242972:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760929.242976:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760929.242980:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760929.242987:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 864 00000100:00080000:3.0:1789760929.242991:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760929.242994:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760929.242998:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760929.243011:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565622784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760929.243020:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760929.243023:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760929.243024:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760929.243028:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 864 00000100:00080000:3.0:1789760929.243031:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760929.243033:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760929.243036:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760929.243044:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565622912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789760929.243101:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565622656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760929.243133:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565622784:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789760929.243153:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565622912:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760929.306813:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760930.330877:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760931.354837:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789760932.315228:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789760932.315323:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789760932.315347:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789760932.315417:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789760932.315433:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789760932.315445:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789760932.315531:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760929/real 1789760932] req@ffff93f844d67100 x1876699565622656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760932.315565:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565622656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760932.315589:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760932.315596:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760932.315602:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565622656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760932.315613:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760929/real 1789760932] req@ffff93f844d67800 x1876699565622784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760932.315637:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565622784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760932.315681:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760932.315687:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760932.315691:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565622784:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789760932.315701:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760929/real 1789760932] req@ffff93f844d67480 x1876699565622912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760932.315717:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565622912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760932.315727:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760932.315732:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760932.315736:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565622912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760932.315745:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760929/real 1789760932] req@ffff93f844d65880 x1876699565622528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760932.315760:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565622528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760932.315770:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760932.315774:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760932.315779:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565622528:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789760932.378845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760933.402834:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760934.362712:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760934.362722:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789760934.362726:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760934.362738:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 869 00000100:00080000:3.0:1789760934.362750:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760934.362757:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789760934.362798:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565623040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760934.362831:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760934.362834:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760934.362835:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760934.362839:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 869 00000100:00080000:3.0:1789760934.362841:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760934.362844:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760934.362853:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760934.362863:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565623168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789760934.362872:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565623040:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789760934.362874:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760934.362884:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760934.362886:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760934.362889:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 869 00000100:00080000:3.0:1789760934.362891:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760934.362894:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760934.362897:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760934.362907:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565623296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760934.362914:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760934.362917:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760934.362918:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760934.362921:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 869 00000100:00080000:3.0:1789760934.362923:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760934.362926:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760934.362929:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760934.362937:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565623424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789760934.362963:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760934.363017:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760934/real 1789760934] req@ffff93f844d64700 x1876699565623040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760934.363038:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565623040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760934.363050:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760934.363054:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760934.363059:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565623040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789760934.363093:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565623168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760934.363119:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760934.363135:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760934/real 1789760934] req@ffff93f844d67b80 x1876699565623168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760934.363149:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565623168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760934.363159:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760934.363162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760934.363165:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565623168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760934.363172:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565623296:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789760934.363195:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760934.363207:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760934/real 1789760934] req@ffff93f844d66680 x1876699565623296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760934.363220:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565623296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760934.363228:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760934.363232:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760934.363235:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565623296:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789760934.363241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565623424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760934.363261:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760934.363273:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760934/real 1789760934] req@ffff93f844d65880 x1876699565623424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760934.363286:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565623424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760934.363294:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760934.363297:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760934.363302:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565623424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760934.426817:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760935.450803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760936.474825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760937.498813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760938.522818:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760939.482732:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760939.482741:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789760939.482745:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760939.482755:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 874 00000100:00080000:3.0:1789760939.482764:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760939.482770:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789760939.482812:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565623552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760939.482835:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760939.482837:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760939.482838:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760939.482841:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 874 00000100:00080000:3.0:1789760939.482843:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760939.482846:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760939.482854:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760939.482863:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565623680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760939.482872:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789760939.482875:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565623552:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789760939.482875:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760939.482876:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760939.482885:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 874 00000100:00080000:3.0:1789760939.482888:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760939.482890:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760939.482893:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760939.482901:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565623808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760939.482913:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760939.482916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760939.482918:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760939.482920:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 874 00000100:00080000:3.0:1789760939.482934:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760939.482936:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760939.482939:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760939.482947:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565623936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789760939.482953:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760939.483006:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760939/real 1789760939] req@ffff93f844d67480 x1876699565623552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760939.483028:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565623552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760939.483039:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760939.483043:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760939.483047:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565623552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789760939.483071:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565623680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760939.483094:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760939.483107:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760939/real 1789760939] req@ffff93f844d67800 x1876699565623680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760939.483120:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565623680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760939.483128:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760939.483132:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760939.483135:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565623680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760939.483141:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565623808:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789760939.483161:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760939.483175:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760939/real 1789760939] req@ffff93f844d67100 x1876699565623808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760939.483188:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565623808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760939.483196:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760939.483199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760939.483202:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565623808:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789760939.483208:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565623936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760939.483226:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760939.483237:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760939/real 1789760939] req@ffff93f844d66a00 x1876699565623936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760939.483247:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565623936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760939.483255:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760939.483258:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760939.483261:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565623936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760939.546800:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760940.570834:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760941.594760:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760942.618798:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760943.642822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760944.602743:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760944.602752:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789760944.602756:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760944.602765:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 879 00000100:00080000:3.0:1789760944.602773:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760944.602778:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789760944.602812:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565624064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760944.602836:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760944.602841:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760944.602844:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760944.602850:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 879 00000100:00080000:3.0:1789760944.602855:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760944.602860:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789760944.602867:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565624064:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789760944.602870:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760944.602887:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565624192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760944.602895:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760944.602898:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760944.602899:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760944.602902:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 879 00000100:00080000:3.0:1789760944.602914:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760944.602917:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760944.602921:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789760944.602926:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760944.602932:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565624320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760944.602942:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760944.602945:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760944.602946:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760944.602949:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 879 00000100:00080000:3.0:1789760944.602953:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760944.602955:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760944.602959:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760944.602966:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565624448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789760944.602990:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760944/real 1789760944] req@ffff93f844d65c00 x1876699565624064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760944.603007:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565624064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760944.603015:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760944.603018:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760944.603021:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565624064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789760944.603039:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565624192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760944.603054:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760944.603062:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760944/real 1789760944] req@ffff93f844d64000 x1876699565624192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760944.603070:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565624192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760944.603075:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760944.603077:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760944.603079:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565624192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760944.603083:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565624320:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789760944.603094:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760944.603100:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760944/real 1789760944] req@ffff93f844d64e00 x1876699565624320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760944.603107:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565624320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760944.603112:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760944.603114:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760944.603116:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565624320:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789760944.603120:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565624448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760944.603130:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760944.603135:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760944/real 1789760944] req@ffff93f844d65500 x1876699565624448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760944.603143:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565624448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760944.603147:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760944.603149:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760944.603151:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565624448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760944.666879:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760945.690836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760946.714824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760947.738813:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760948.762804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760949.722716:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760949.722724:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789760949.722727:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760949.722735:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 884 00000100:00080000:3.0:1789760949.722743:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760949.722748:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789760949.722781:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565624576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760949.722802:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760949.722805:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760949.722806:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760949.722809:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 884 00000100:00080000:3.0:1789760949.722812:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760949.722815:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760949.722822:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760949.722831:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565624704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789760949.722838:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565624576:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789760949.722846:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760949.722851:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760949.722853:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760949.722858:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 884 00000100:00080000:3.0:1789760949.722870:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760949.722875:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760949.722882:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760949.722893:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565624832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760949.722900:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760949.722903:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760949.722904:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760949.722907:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 884 00000100:00080000:3.0:1789760949.722910:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760949.722913:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760949.722916:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760949.722923:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565624960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789760949.722943:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565624704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760949.722968:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565624832:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789760949.722984:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565624960:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760949.786836:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760950.810831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760951.834808:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789760952.794913:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789760952.794948:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789760952.794962:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789760952.794996:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789760952.795013:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789760952.795025:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789760952.795047:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760949/real 1789760952] req@ffff93f844d65f80 x1876699565624704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760952.795079:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565624704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760952.795093:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760952.795099:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760952.795104:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565624704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760952.795115:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760949/real 1789760952] req@ffff93f844d66d80 x1876699565624832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760952.795129:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565624832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760952.795139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760952.795144:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760952.795148:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565624832:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789760952.795157:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760949/real 1789760952] req@ffff93f844d64a80 x1876699565624960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760952.795170:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565624960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760952.795180:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760952.795184:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760952.795187:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565624960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760952.795195:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760949/real 1789760952] req@ffff93f844d64380 x1876699565624576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760952.795231:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565624576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760952.795240:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760952.795244:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760952.795248:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565624576:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789760952.858854:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760953.882782:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760954.842723:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760954.842734:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789760954.842740:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760954.842752:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 889 00000100:00080000:3.0:1789760954.842761:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760954.842770:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789760954.842809:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565625088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760954.842837:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760954.842840:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760954.842842:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760954.842847:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 889 00000100:00080000:3.0:1789760954.842850:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760954.842853:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760954.842862:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760954.842873:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565625216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760954.842881:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760954.842885:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760954.842887:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760954.842891:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 889 00000100:00080000:3.0:1789760954.842895:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760954.842899:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760954.842905:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760954.842920:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565625344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760954.842939:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760954.842943:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760954.842945:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760954.842951:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 889 00000100:00080000:3.0:1789760954.842955:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760954.842960:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760954.842966:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760954.842981:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565625472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789760954.843022:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565625088:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1789760954.843107:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789760954.843165:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760954/real 1789760954] req@ffff93f844d66300 x1876699565625088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760954.843201:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565625088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760954.843214:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760954.843219:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760954.843223:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565625088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789760954.843232:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565625216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789760954.843259:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789760954.843281:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760954/real 1789760954] req@ffff93f844d65180 x1876699565625216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760954.843296:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565625216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760954.843305:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760954.843310:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760954.843313:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565625216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789760954.843320:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565625344:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789760954.843346:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789760954.843359:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760954/real 1789760954] req@ffff93f844d64380 x1876699565625344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760954.843371:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565625344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760954.843379:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760954.843383:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760954.843386:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565625344:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789760954.843392:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565625472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789760954.843413:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789760954.843425:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760954/real 1789760954] req@ffff93f844d64a80 x1876699565625472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760954.843436:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565625472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760954.843444:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760954.843447:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760954.843450:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565625472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760954.906827:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760955.930823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760956.954790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760957.978765:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760959.002805:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789760959.962744:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760959.962757:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789760959.962762:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760959.962774:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 894 00000100:00080000:3.0:1789760959.962795:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760959.962803:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789760959.962849:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565625600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760959.962876:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760959.962879:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760959.962882:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760959.962885:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 894 00000100:00080000:3.0:1789760959.962888:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760959.962891:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760959.962901:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760959.962915:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565625728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760959.962923:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789760959.962929:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565625600:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789760959.962929:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789760959.962933:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760959.962940:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 894 00000100:00080000:3.0:1789760959.962943:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760959.962945:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760959.962949:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789760959.962958:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565625856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789760959.962966:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789760959.962969:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789760959.962971:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789760959.962974:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 894 00000100:00080000:3.0:1789760959.962976:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789760959.962978:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789760959.962982:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760959.962989:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565625984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789760959.963008:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789760959.963059:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760959/real 1789760959] req@ffff93f844d66d80 x1876699565625600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760959.963080:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565625600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760959.963089:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760959.963092:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760959.963096:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565625600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789760959.963121:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565625728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789760959.963136:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789760959.963145:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760959/real 1789760959] req@ffff93f844d65f80 x1876699565625728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760959.963173:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565625728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760959.963180:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760959.963182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760959.963184:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565625728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789760959.963189:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565625856:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789760959.963211:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789760959.963224:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760959/real 1789760959] req@ffff93f844d65500 x1876699565625856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760959.963237:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565625856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760959.963254:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760959.963256:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760959.963259:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565625856:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789760959.963263:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565625984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789760959.963279:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789760959.963287:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760959/real 1789760959] req@ffff93f844d64e00 x1876699565625984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789760959.963295:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565625984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789760959.963301:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789760959.963303:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789760959.963305:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565625984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760960.026902:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760961.050808:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760962.074878:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760963.098925:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760964.122865:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789760965.083070:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760965.083082:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789760965.083087:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760965.083099:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 900 00000100:00080000:0.0:1789760965.083109:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760965.083130:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789760965.083174:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565626112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760965.083204:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760965.083209:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760965.083211:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760965.083215:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 900 00000100:00080000:0.0:1789760965.083220:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760965.083224:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760965.083236:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760965.083252:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565626240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760965.083263:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760965.083266:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789760965.083268:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760965.083271:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 900 00000100:00080000:0.0:1789760965.083274:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760965.083277:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760965.083289:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789760965.083304:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565626112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789760965.083307:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565626368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760965.083331:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760965.083335:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760965.083337:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760965.083342:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 900 00000100:00080000:0.0:1789760965.083346:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760965.083352:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760965.083359:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760965.083375:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565626496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789760965.083432:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760965.083482:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760965/real 1789760965] req@ffff93f850d40e00 x1876699565626112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760965.083499:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565626112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760965.083508:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760965.083512:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760965.083515:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565626112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789760965.083522:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565626240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760965.083539:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760965.083548:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760965/real 1789760965] req@ffff93f850d41c00 x1876699565626240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760965.083557:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565626240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760965.083562:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760965.083564:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760965.083566:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565626240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760965.083598:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565626368:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789760965.083621:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760965.083634:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760965/real 1789760965] req@ffff93f850d40700 x1876699565626368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760965.083705:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565626368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760965.083719:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760965.083723:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760965.083726:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565626368:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789760965.083734:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565626496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760965.083754:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760965.083763:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760965/real 1789760965] req@ffff93f850d42680 x1876699565626496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760965.083771:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565626496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760965.083777:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760965.083779:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760965.083781:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565626496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760965.146872:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760966.170838:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760967.194775:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760968.218855:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760969.242835:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789760970.202780:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760970.202788:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789760970.202791:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760970.202800:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 905 00000100:00080000:0.0:1789760970.202809:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760970.202814:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789760970.202850:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565626624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760970.202891:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760970.202894:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760970.202896:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760970.202899:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 905 00000100:00080000:0.0:1789760970.202902:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760970.202906:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760970.202917:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760970.202929:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565626752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789760970.202941:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565626624:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789760970.202944:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760970.202964:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789760970.202966:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760970.202971:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 905 00000100:00080000:0.0:1789760970.202974:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760970.202976:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760970.202980:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789760970.202991:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565626880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760970.202998:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760970.203001:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760970.203003:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760970.203006:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 905 00000100:00080000:0.0:1789760970.203009:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760970.203012:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760970.203017:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760970.203024:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565627008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789760970.203045:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760970.203109:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760970/real 1789760970] req@ffff93f850d40000 x1876699565626624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760970.203137:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565626624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789760995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760970.203160:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760970.203164:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760970.203167:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565626624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789760970.203193:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565626752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760970.203213:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760970.203222:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760970/real 1789760970] req@ffff93f850d41500 x1876699565626752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760970.203241:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565626752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760970.203247:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760970.203249:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760970.203251:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565626752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760970.203255:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565626880:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789760970.203269:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760970.203275:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760970/real 1789760970] req@ffff93f850d43100 x1876699565626880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760970.203282:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565626880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789760995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760970.203288:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760970.203290:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760970.203291:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565626880:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789760970.203296:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565627008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760970.203308:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760970.203314:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760970/real 1789760970] req@ffff93f850d41f80 x1876699565627008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760970.203321:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565627008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789760995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760970.203326:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760970.203328:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760970.203330:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565627008:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760970.266874:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760971.290813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760972.315078:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760973.338836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760974.362836:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789760975.322805:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760975.322820:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789760975.322826:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760975.322853:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 910 00000100:00080000:0.0:1789760975.322867:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760975.322876:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789760975.322931:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565627136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760975.322965:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760975.322972:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760975.322975:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760975.322979:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 910 00000100:00080000:0.0:1789760975.322999:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760975.323003:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760975.323013:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760975.323027:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565627264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760975.323056:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760975.323060:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789760975.323061:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760975.323066:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 910 00000100:00080000:0.0:1789760975.323070:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760975.323077:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760975.323085:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789760975.323090:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565627136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789760975.323104:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565627392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760975.323143:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760975.323148:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760975.323152:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760975.323158:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 910 00000100:00080000:0.0:1789760975.323165:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760975.323173:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760975.323182:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760975.323206:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565627520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789760975.323237:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760975.323329:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760975/real 1789760975] req@ffff93f850d43480 x1876699565627136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760975.323359:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565627136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760975.323376:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760975.323382:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760975.323388:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565627136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789760975.323399:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565627264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760975.323435:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760975.323455:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760975/real 1789760975] req@ffff93f850d43800 x1876699565627264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760975.323475:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565627264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760975.323486:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760975.323491:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760975.323510:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565627264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760975.323556:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565627392:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789760975.323582:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760975.323596:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760975/real 1789760975] req@ffff93f850d41180 x1876699565627392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760975.323609:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565627392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760975.323618:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760975.323622:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760975.323625:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565627392:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789760975.323632:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565627520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760975.323654:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760975.323697:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760975/real 1789760975] req@ffff93f850d41f80 x1876699565627520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760975.323711:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565627520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760975.323738:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760975.323742:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760975.323745:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565627520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760975.386852:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760976.410804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760977.434895:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760978.458795:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760979.482796:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789760980.442717:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760980.442726:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789760980.442729:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760980.442738:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 915 00000100:00080000:0.0:1789760980.442745:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760980.442751:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789760980.442786:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565627648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760980.442810:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760980.442813:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760980.442814:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760980.442817:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 915 00000100:00080000:0.0:1789760980.442819:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760980.442822:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760980.442829:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760980.442837:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565627776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760980.442844:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760980.442846:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789760980.442847:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760980.442849:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 915 00000100:00080000:0.0:1789760980.442852:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760980.442854:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760980.442856:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789760980.442862:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565627904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760980.442870:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760980.442872:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760980.442873:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760980.442876:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 915 00000100:00080000:0.0:1789760980.442877:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760980.442880:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760980.442882:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760980.442905:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565628032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789760980.442917:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565627648:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1789760980.443012:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760980.443066:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760980/real 1789760980] req@ffff93f850d43100 x1876699565627648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760980.443080:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565627648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760980.443088:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760980.443091:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760980.443094:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565627648:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789760980.443099:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565627776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760980.443115:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760980.443123:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760980/real 1789760980] req@ffff93f850d41500 x1876699565627776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760980.443131:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565627776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760980.443136:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760980.443138:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760980.443140:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565627776:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789760980.443144:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565627904:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789760980.443155:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760980.443160:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760980/real 1789760980] req@ffff93f850d40000 x1876699565627904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760980.443168:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565627904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760980.443173:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760980.443175:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760980.443177:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565627904:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789760980.443196:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565628032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789760980.443207:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789760980.443213:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760980/real 1789760980] req@ffff93f850d42680 x1876699565628032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760980.443220:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565628032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789760980.443225:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789760980.443226:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789760980.443228:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565628032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760980.506990:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760981.530919:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760982.554787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760983.578854:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760984.602836:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789760985.562771:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760985.562780:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789760985.562784:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760985.562796:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 920 00000100:00080000:0.0:1789760985.562806:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760985.562814:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789760985.562852:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565628160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760985.562878:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760985.562883:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760985.562885:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760985.562888:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 920 00000100:00080000:0.0:1789760985.562891:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760985.562894:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760985.562903:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760985.562915:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565628288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760985.562926:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760985.562932:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789760985.562934:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1789760985.562934:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565628160:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789760985.562947:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 920 00000100:00080000:0.0:1789760985.562952:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760985.562956:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760985.562976:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789760985.562990:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565628416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789760985.563016:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789760985.563019:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789760985.563021:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789760985.563027:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 920 00000100:00080000:0.0:1789760985.563031:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789760985.563035:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789760985.563040:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789760985.563050:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42a00 x1876699565628544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789760985.563108:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565628288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789760985.563149:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565628416:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789760985.563172:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565628544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760985.626829:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760986.650829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760987.674798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789760988.635123:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789760988.635219:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789760988.635246:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789760988.635311:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789760988.635335:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789760988.635351:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1789760988.635393:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760985/real 1789760988] req@ffff93f850d41c00 x1876699565628288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760988.635432:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565628288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760988.635446:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760988.635452:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760988.635458:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565628288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789760988.635468:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760985/real 1789760988] req@ffff93f850d40e00 x1876699565628416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760988.635482:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565628416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760988.635491:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760988.635495:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760988.635499:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565628416:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789760988.635507:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760985/real 1789760988] req@ffff93f850d42a00 x1876699565628544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760988.635527:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42a00 x1876699565628544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760988.635535:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760988.635538:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760988.635542:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565628544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789760988.635549:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760985/real 1789760988] req@ffff93f850d40700 x1876699565628160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760988.635561:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565628160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760988.635569:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760988.635573:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760988.635576:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565628160:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789760988.698869:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760989.722795:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789760990.682775:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760990.682783:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789760990.682786:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760990.682795:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 925 00000100:00080000:2.0:1789760990.682804:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760990.682811:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789760990.682845:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565628672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789760990.682871:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760990.682874:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789760990.682876:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760990.682880:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 925 00000100:00080000:2.0:1789760990.682882:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760990.682885:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789760990.682893:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789760990.682907:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565628800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789760990.682908:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565628672:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789760990.682942:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760990.682946:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789760990.682948:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760990.682952:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 925 00000100:00080000:2.0:1789760990.682956:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760990.682959:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789760990.682964:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789760990.682973:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565628928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789760990.682987:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789760990.682996:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789760990.683001:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789760990.683003:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789760990.683008:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 925 00000100:00080000:2.0:1789760990.683013:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789760990.683018:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789760990.683024:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789760990.683034:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760990/real 1789760990] req@ffff93f8517c0380 x1876699565628672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789760990.683040:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565629056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789760990.683075:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565628672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760990.683087:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760990.683092:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760990.683096:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565628672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789760990.683122:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565628800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789760990.683155:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789760990.683169:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760990/real 1789760990] req@ffff93f8517c1500 x1876699565628800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760990.683188:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565628800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760990.683197:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760990.683201:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760990.683205:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565628800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789760990.683211:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565628928:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789760990.683233:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789760990.683245:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760990/real 1789760990] req@ffff93f8517c0a80 x1876699565628928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760990.683256:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565628928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760990.683265:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760990.683267:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760990.683271:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565628928:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789760990.683276:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565629056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789760990.683313:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789760990.683325:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760990/real 1789760990] req@ffff93f8517c2680 x1876699565629056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760990.683336:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565629056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789760990.683349:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789760990.683353:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789760990.683356:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565629056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760990.746768:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760991.770900:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760992.794802:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760993.818884:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760994.842786:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789760995.802791:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760995.802805:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789760995.802811:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789760995.802826:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 930 00000100:00080000:1.0:1789760995.802840:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789760995.802850:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789760995.802908:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565629184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789760995.802939:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760995.802944:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789760995.802946:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789760995.802951:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 930 00000100:00080000:1.0:1789760995.802957:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789760995.802963:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789760995.802976:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760995.802986:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565629184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789760995.802993:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565629312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789760995.803005:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760995.803010:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789760995.803013:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789760995.803020:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 930 00000100:00080000:1.0:1789760995.803026:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789760995.803032:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789760995.803040:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789760995.803055:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565629440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789760995.803065:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789760995.803083:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789760995.803087:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789760995.803088:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789760995.803094:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 930 00000100:00080000:1.0:1789760995.803098:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789760995.803103:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789760995.803109:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789760995.803115:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760995/real 1789760995] req@ffff93f844cc9c00 x1876699565629184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789760995.803119:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565629568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789760995.803193:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565629184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760995.803205:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760995.803208:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760995.803211:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565629184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789760995.803231:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565629312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789760995.803248:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760995.803257:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760995/real 1789760995] req@ffff93f844ccb100 x1876699565629312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760995.803265:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565629312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760995.803270:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760995.803272:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760995.803274:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565629312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789760995.803278:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565629440:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789760995.803289:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760995.803295:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760995/real 1789760995] req@ffff93f844ccb800 x1876699565629440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760995.803302:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565629440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760995.803312:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760995.803314:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760995.803316:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565629440:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789760995.803319:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565629568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789760995.803329:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789760995.803335:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789760995/real 1789760995] req@ffff93f844cc8000 x1876699565629568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789760995.803342:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565629568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789760995.803347:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789760995.803349:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789760995.803351:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565629568:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789760995.866856:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760996.890797:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760997.914784:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760998.938836:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760999.962828:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761000.922759:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761000.922768:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761000.922772:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761000.922780:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 935 00000100:00080000:0.0:1789761000.922790:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761000.922796:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761000.922829:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565629696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761000.922864:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761000.922868:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761000.922869:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761000.922872:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 935 00000100:00080000:0.0:1789761000.922875:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761000.922878:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761000.922888:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761000.922904:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565629824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761000.922920:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565629696:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761000.922922:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761000.922936:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761000.922939:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761000.922944:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 935 00000100:00080000:0.0:1789761000.922947:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761000.922950:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761000.922955:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761000.922966:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565629952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761000.922979:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761000.922983:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761000.922985:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761000.922988:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 935 00000100:00080000:0.0:1789761000.922991:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761000.922994:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761000.922999:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761000.923012:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565630080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761000.923012:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761000.923073:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761000/real 1789761000] req@ffff93f850d42300 x1876699565629696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761000.923098:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565629696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761000.923112:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761000.923117:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761000.923121:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565629696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761000.923152:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565629824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761000.923179:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761000.923193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761000/real 1789761000] req@ffff93f850d40380 x1876699565629824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761000.923207:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565629824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761000.923216:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761000.923220:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761000.923224:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565629824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761000.923232:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565629952:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761000.923251:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761000.923260:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761000/real 1789761000] req@ffff93f850d43b80 x1876699565629952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761000.923268:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565629952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761000.923283:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761000.923285:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761000.923288:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565629952:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761000.923293:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565630080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761000.923307:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761000.923314:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761000/real 1789761000] req@ffff93f850d40700 x1876699565630080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761000.923323:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565630080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761000.923330:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761000.923334:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761000.923337:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565630080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761000.986902:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761002.010778:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761003.034787:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761004.058762:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761005.082964:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761006.042776:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761006.042784:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761006.042788:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761006.042796:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 941 00000100:00080000:2.0:1789761006.042805:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761006.042832:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761006.042868:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565630208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761006.042891:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761006.042894:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761006.042895:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761006.042898:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 941 00000100:00080000:2.0:1789761006.042901:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761006.042904:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761006.042914:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761006.042923:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565630208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761006.042925:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565630336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761006.042938:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761006.042941:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761006.042942:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761006.042945:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 941 00000100:00080000:2.0:1789761006.042946:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761006.042949:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761006.042952:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761006.042960:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565630464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761006.042967:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761006.042969:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761006.042971:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761006.042973:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 941 00000100:00080000:2.0:1789761006.042976:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761006.042979:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761006.042982:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761006.042992:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565630592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761006.042992:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761006.043031:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761006/real 1789761006] req@ffff93f8517c1180 x1876699565630208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761006.043045:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565630208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761006.043053:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761006.043056:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761006.043059:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565630208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761006.043077:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565630336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761006.043092:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761006.043100:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761006/real 1789761006] req@ffff93f8517c3100 x1876699565630336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761006.043108:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565630336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761006.043113:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761006.043115:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761006.043117:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565630336:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761006.043120:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565630464:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761006.043131:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761006.043137:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761006/real 1789761006] req@ffff93f8517c2a00 x1876699565630464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761006.043144:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565630464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761006.043149:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761006.043151:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761006.043152:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565630464:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761006.043156:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565630592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761006.043165:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761006.043170:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761006/real 1789761006] req@ffff93f8517c2d80 x1876699565630592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761006.043178:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565630592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761006.043183:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761006.043185:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761006.043186:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565630592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761006.106843:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761007.130841:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761008.154788:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761009.178811:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761010.202766:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761011.162755:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761011.162777:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761011.162784:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761011.162795:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 946 00000100:00080000:2.0:1789761011.162807:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761011.162814:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761011.162856:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565630720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761011.162887:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761011.162892:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761011.162894:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761011.162898:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 946 00000100:00080000:2.0:1789761011.162901:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761011.162906:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761011.162916:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761011.162934:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565630848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761011.162947:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761011.162952:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761011.162955:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761011.162963:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 946 00000100:00100000:3.0:1789761011.162963:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565630720:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761011.162973:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761011.162981:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761011.162989:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761011.163006:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565630976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761011.163024:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761011.163028:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761011.163031:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761011.163035:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 946 00000100:00080000:2.0:1789761011.163041:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761011.163046:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761011.163052:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761011.163073:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565631104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761011.163074:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761011.163140:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761011/real 1789761011] req@ffff93f8517c1880 x1876699565630720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761011.163162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565630720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761011.163174:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761011.163179:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761011.163184:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565630720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761011.163215:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565630848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761011.163240:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761011.163253:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761011/real 1789761011] req@ffff93f8517c3480 x1876699565630848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761011.163267:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565630848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761011.163276:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761011.163279:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761011.163282:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565630848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761011.163289:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565630976:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761011.163310:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761011.163320:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761011/real 1789761011] req@ffff93f8517c3800 x1876699565630976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761011.163332:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565630976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761011.163341:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761011.163344:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761011.163348:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565630976:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761011.163355:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565631104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761011.163373:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761011.163383:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761011/real 1789761011] req@ffff93f8517c0e00 x1876699565631104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761011.163394:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565631104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761011.163409:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761011.163412:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761011.163416:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565631104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761011.226841:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761012.250787:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761013.274789:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761014.298791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761015.322806:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761016.282805:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761016.282819:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761016.282826:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761016.282840:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 951 00000100:00080000:3.0:1789761016.282853:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761016.282865:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761016.282916:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565631232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761016.282952:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761016.282957:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761016.282958:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761016.282964:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 951 00000100:00080000:3.0:1789761016.282968:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761016.282973:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761016.282985:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761016.283002:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565631360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761016.283009:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565631232:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761016.283024:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761016.283028:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761016.283030:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761016.283036:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 951 00000100:00080000:3.0:1789761016.283040:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761016.283044:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761016.283050:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761016.283066:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565631488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761016.283078:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761016.283083:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761016.283086:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761016.283090:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 951 00000100:00080000:3.0:1789761016.283095:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1789761016.283100:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761016.283105:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761016.283112:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761016.283138:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565631616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761016.283162:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761016/real 1789761016] req@ffff93f844d64000 x1876699565631232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761016.283187:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565631232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761016.283201:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761016.283207:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761016.283211:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565631232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761016.283242:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565631360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761016.283269:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761016.283284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761016/real 1789761016] req@ffff93f844d65c00 x1876699565631360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761016.283298:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565631360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761016.283307:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761016.283311:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761016.283315:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565631360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761016.283322:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565631488:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761016.283349:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761016.283362:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761016/real 1789761016] req@ffff93f844d66a00 x1876699565631488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761016.283375:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565631488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761016.283384:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761016.283388:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761016.283392:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565631488:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761016.283398:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565631616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761016.283422:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761016.283435:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761016/real 1789761016] req@ffff93f844d67100 x1876699565631616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761016.283449:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565631616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761016.283457:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761016.283461:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761016.283465:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565631616:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761016.346849:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761017.370842:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761018.394868:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761019.418807:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761020.442904:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761021.402741:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761021.402752:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761021.402757:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761021.402767:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 956 00000100:00080000:1.0:1789761021.402778:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761021.402785:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761021.402830:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565631744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761021.402863:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761021.402867:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761021.402869:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761021.402873:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 956 00000100:00080000:1.0:1789761021.402877:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761021.402882:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761021.402891:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761021.402904:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565631744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761021.402905:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565631872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761021.402920:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761021.402922:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761021.402924:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761021.402927:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 956 00000100:00080000:1.0:1789761021.402929:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761021.402932:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761021.402935:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761021.402943:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565632000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761021.402952:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761021.402955:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761021.402956:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761021.402958:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 956 00000100:00080000:1.0:1789761021.402963:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761021.402966:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761021.402968:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761021.402975:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565632128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761021.403000:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761021.403056:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761021/real 1789761021] req@ffff93f844ccaa00 x1876699565631744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761021.403080:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565631744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761021.403094:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761021.403099:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761021.403104:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565631744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761021.403133:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565631872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761021.403154:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761021.403165:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761021/real 1789761021] req@ffff93f844ccb480 x1876699565631872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761021.403179:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565631872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761021.403187:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761021.403191:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761021.403195:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565631872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761021.403203:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565632000:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761021.403226:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761021.403236:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761021/real 1789761021] req@ffff93f844cc8700 x1876699565632000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761021.403249:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565632000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761021.403256:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761021.403258:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761021.403260:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565632000:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761021.403264:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565632128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761021.403277:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761021.403283:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761021/real 1789761021] req@ffff93f844ccbb80 x1876699565632128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761021.403291:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565632128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761021.403296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761021.403298:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761021.403300:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565632128:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761021.466805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761022.490885:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761023.514822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761024.538884:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761025.562853:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761026.522789:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761026.522804:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761026.522810:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761026.522826:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 961 00000100:00080000:3.0:1789761026.522838:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761026.522849:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761026.522891:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565632256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761026.522924:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761026.522929:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761026.522931:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761026.522938:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 961 00000100:00080000:3.0:1789761026.522942:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761026.522948:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761026.522960:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761026.522978:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565632384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761026.522991:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761026.522995:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761026.522997:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761026.523002:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 961 00000100:00080000:3.0:1789761026.523007:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761026.523012:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761026.523019:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761026.523029:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565632256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761026.523038:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565632512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761026.523108:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761026.523137:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761026.523141:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761026.523143:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761026.523149:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 961 00000100:00080000:3.0:1789761026.523155:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789761026.523161:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761026/real 1789761026] req@ffff93f844d67800 x1876699565632256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761026.523165:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761026.523177:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761026.523190:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565632256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761026.523195:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565632640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761026.523212:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761026.523223:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761026.523229:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565632256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761026.523238:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565632384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761026.523262:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761026.523275:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761026/real 1789761026] req@ffff93f844d67480 x1876699565632384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761026.523289:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565632384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761026.523298:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761026.523301:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761026.523305:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565632384:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761026.523343:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565632512:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761026.523366:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761026.523378:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761026/real 1789761026] req@ffff93f844d65880 x1876699565632512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761026.523392:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565632512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761026.523409:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761026.523413:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761026.523416:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565632512:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761026.523423:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565632640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761026.523442:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761026.523456:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761026/real 1789761026] req@ffff93f844d66680 x1876699565632640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761026.523469:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565632640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761026.523480:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761026.523484:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761026.523488:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565632640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761026.586905:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761027.610767:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761028.634829:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761029.658827:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761030.682791:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761031.643340:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761031.643352:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761031.643356:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761031.643369:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 966 00000100:00080000:3.0:1789761031.643379:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761031.643387:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761031.643423:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565632768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761031.643444:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761031.643447:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761031.643449:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761031.643452:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 966 00000100:00080000:3.0:1789761031.643454:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761031.643458:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761031.643467:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761031.643476:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565632896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761031.643484:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565632768:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761031.643486:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761031.643493:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761031.643495:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761031.643498:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 966 00000100:00080000:3.0:1789761031.643501:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761031.643503:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761031.643506:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761031.643516:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565633024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761031.643524:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761031.643527:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761031.643529:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761031.643531:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 966 00000100:00080000:3.0:1789761031.643534:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761031.643536:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761031.643539:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761031.643548:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565633152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761031.643556:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761031.643598:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761031/real 1789761031] req@ffff93f844d67b80 x1876699565632768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761031.643626:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565632768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761031.643638:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761031.643676:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761031.643681:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565632768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761031.643703:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565632896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761031.643723:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761031.643734:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761031/real 1789761031] req@ffff93f844d64700 x1876699565632896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761031.643743:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565632896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761031.643748:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761031.643751:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761031.643753:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565632896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761031.643757:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565633024:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761031.643768:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761031.643775:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761031/real 1789761031] req@ffff93f844d66680 x1876699565633024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761031.643783:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565633024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761031.643790:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761031.643792:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761031.643794:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565633024:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761031.643804:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565633152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761031.643821:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761031.643828:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761031/real 1789761031] req@ffff93f844d65880 x1876699565633152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761031.643839:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565633152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761031.643844:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761031.643846:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761031.643848:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565633152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761031.706808:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761032.730806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761033.754803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761034.778803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761035.802814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761036.762765:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761036.762776:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761036.762781:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761036.762794:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 971 00000100:00080000:1.0:1789761036.762808:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761036.762818:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761036.762869:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565633280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761036.762901:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761036.762905:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761036.762908:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761036.762914:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 971 00000100:00080000:1.0:1789761036.762919:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761036.762924:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761036.762931:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565633280:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789761036.762936:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761036.762957:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565633408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761036.762970:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761036.762974:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761036.762977:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761036.762981:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 971 00000100:00080000:1.0:1789761036.762994:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761036.762999:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789761036.763003:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789761036.763010:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761036.763025:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565633536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761036.763038:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761036.763041:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761036.763043:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761036.763047:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 971 00000100:00080000:1.0:1789761036.763050:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761036.763054:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761036.763057:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761036.763061:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761036/real 1789761036] req@ffff93f844cc8a80 x1876699565633280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761036.763068:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565633664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761036.763095:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565633280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761036.763107:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761036.763111:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761036.763114:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565633280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761036.763137:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565633408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761036.763164:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761036.763175:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761036/real 1789761036] req@ffff93f844cc8380 x1876699565633408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761036.763184:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565633408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761036.763205:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761036.763207:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761036.763210:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565633408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761036.763214:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565633536:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761036.763226:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761036.763233:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761036/real 1789761036] req@ffff93f844cca300 x1876699565633536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761036.763241:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565633536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761036.763246:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761036.763248:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761036.763250:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565633536:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761036.763254:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565633664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761036.763265:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761036.763272:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761036/real 1789761036] req@ffff93f844cc9180 x1876699565633664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761036.763280:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565633664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761036.763284:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761036.763286:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761036.763289:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565633664:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761036.826774:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761037.850817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761038.874810:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761039.898864:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761040.922780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761041.882771:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761041.882781:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761041.882785:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761041.882797:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 976 00000100:00080000:2.0:1789761041.882807:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761041.882815:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761041.882860:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565633792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761041.882887:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761041.882890:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761041.882892:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761041.882895:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 976 00000100:00080000:2.0:1789761041.882897:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761041.882900:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761041.882911:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761041.882920:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565633792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761041.882922:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565633920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761041.882934:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761041.882937:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761041.882938:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761041.882941:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 976 00000100:00080000:2.0:1789761041.882942:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761041.882944:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761041.882947:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761041.882956:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565634048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761041.882962:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761041.882964:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761041.882965:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761041.882968:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 976 00000100:00080000:2.0:1789761041.882971:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761041.882974:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761041.882977:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761041.882985:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565634176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761041.883016:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761041.883067:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761041/real 1789761041] req@ffff93f8517c0e00 x1876699565633792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761041.883085:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565633792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761041.883095:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761041.883099:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761041.883102:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565633792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761041.883128:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565633920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761041.883147:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761041.883157:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761041/real 1789761041] req@ffff93f8517c3800 x1876699565633920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761041.883166:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565633920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761041.883172:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761041.883174:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761041.883177:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565633920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761041.883180:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565634048:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761041.883192:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761041.883199:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761041/real 1789761041] req@ffff93f8517c3480 x1876699565634048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761041.883207:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565634048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761041.883213:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761041.883216:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761041.883219:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565634048:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761041.883225:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565634176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761041.883245:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761041.883258:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761041/real 1789761041] req@ffff93f8517c1880 x1876699565634176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761041.883271:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565634176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761041.883280:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761041.883284:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761041.883287:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565634176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761041.946840:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761042.970782:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761043.994826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761045.018770:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761046.042792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761047.002745:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761047.002752:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761047.002757:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761047.002766:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 981 00000100:00080000:2.0:1789761047.002774:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761047.002781:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761047.002823:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565634304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761047.002843:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761047.002845:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761047.002847:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761047.002849:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 981 00000100:00080000:2.0:1789761047.002851:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761047.002855:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761047.002863:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761047.002871:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565634432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761047.002878:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761047.002882:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1789761047.002884:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565634304:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761047.002884:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761047.002889:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 981 00000100:00080000:2.0:1789761047.002896:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761047.002898:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761047.002901:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761047.002909:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565634560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761047.002920:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761047.002923:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761047.002924:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761047.002926:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 981 00000100:00080000:2.0:1789761047.002929:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761047.002931:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761047.002934:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761047.002940:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565634688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761047.002962:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761047.003000:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761047/real 1789761047] req@ffff93f8517c2d80 x1876699565634304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761047.003014:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565634304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761047.003022:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761047.003026:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761047.003029:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565634304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761047.003048:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565634432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761047.003064:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761047.003072:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761047/real 1789761047] req@ffff93f8517c2a00 x1876699565634432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761047.003082:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565634432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761047.003090:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761047.003093:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761047.003096:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565634432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761047.003112:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565634560:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761047.003132:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761047.003143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761047/real 1789761047] req@ffff93f8517c3100 x1876699565634560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761047.003154:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565634560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761047.003162:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761047.003166:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761047.003169:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565634560:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761047.003175:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565634688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761047.003195:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761047.003206:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761047/real 1789761047] req@ffff93f8517c1180 x1876699565634688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761047.003218:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565634688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761047.003225:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761047.003228:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761047.003231:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565634688:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761047.066793:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761048.090789:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761049.114765:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761050.138843:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761051.162796:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761052.122716:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761052.122725:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761052.122730:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761052.122738:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 987 00000100:00080000:2.0:1789761052.122748:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761052.122755:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761052.122790:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565634816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761052.122816:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761052.122820:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761052.122822:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761052.122827:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 987 00000100:00080000:2.0:1789761052.122832:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761052.122838:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761052.122851:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761052.122869:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565634944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761052.122872:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565634816:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761052.122882:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761052.122887:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761052.122889:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761052.122892:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 987 00000100:00080000:2.0:1789761052.122895:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761052.122897:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761052.122901:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761052.122911:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565635072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761052.122918:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761052.122920:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761052.122921:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761052.122924:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 987 00000100:00080000:2.0:1789761052.122926:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761052.122929:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761052.122931:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761052.122940:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565635200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761052.123028:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565634944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761052.123060:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565635072:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761052.123090:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565635200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761052.186777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761053.210779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761054.234769:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789761055.194987:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789761055.195019:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789761055.196099:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761055.196151:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761055.196165:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761055.196179:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789761055.196209:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761052/real 1789761055] req@ffff93f8517c0a80 x1876699565634944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761055.196237:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565634944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761055.196250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761055.196255:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761055.196260:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565634944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761055.196269:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761052/real 1789761055] req@ffff93f8517c1500 x1876699565635072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761055.196284:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565635072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761055.196292:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761055.196296:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761055.196300:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565635072:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761055.196307:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761052/real 1789761055] req@ffff93f8517c0380 x1876699565635200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761055.196319:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565635200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761055.196326:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761055.196330:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761055.196333:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565635200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761055.196340:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761052/real 1789761055] req@ffff93f8517c2680 x1876699565634816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761055.196352:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565634816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761055.196361:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761055.196364:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761055.196368:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565634816:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761055.258779:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761056.282796:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761057.242836:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761057.242851:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761057.242865:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761057.242877:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 992 00000100:00080000:2.0:1789761057.242890:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761057.242899:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761057.242943:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565635328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761057.242974:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761057.242979:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761057.242981:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761057.242986:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 992 00000100:00080000:2.0:1789761057.242991:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761057.242996:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761057.243008:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761057.243030:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565635328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761057.243032:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565635456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761057.243048:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761057.243053:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761057.243055:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761057.243060:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 992 00000100:00080000:2.0:1789761057.243064:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761057.243069:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761057.243075:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761057.243090:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565635584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761057.243110:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761057.243114:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761057.243117:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761057.243121:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 992 00000100:00080000:2.0:1789761057.243126:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761057.243134:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1789761057.243135:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1789761057.243149:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761057.243171:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565635712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761057.243201:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761057/real 1789761057] req@ffff93f8517c2300 x1876699565635328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761057.243236:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565635328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761057.243249:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761057.243253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761057.243258:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565635328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761057.243291:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565635456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761057.243317:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761057.243332:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761057/real 1789761057] req@ffff93f8517c1f80 x1876699565635456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761057.243346:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565635456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761057.243354:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761057.243359:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761057.243362:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565635456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761057.243388:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565635584:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761057.243415:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761057.243443:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761057/real 1789761057] req@ffff93f8517c3b80 x1876699565635584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761057.243458:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565635584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761057.243467:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761057.243471:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761057.243474:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565635584:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761057.243481:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565635712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761057.243505:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761057.243518:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761057/real 1789761057] req@ffff93f8517c0000 x1876699565635712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761057.243532:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565635712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761057.243540:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761057.243544:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761057.243547:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565635712:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761057.306847:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761058.330766:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761059.354832:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761060.378765:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789761061.402798:0:11738:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 3/3 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789761061.402879:0:5443:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 3 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789761061.402882:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761061.402892:0:5443:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff93f851b31000 lock: ffff93f8517bb400/0x4e3cc7995dd80f1e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x721975f94672fee6 expref: -99 pid: 11706 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1789761061.402927:0:5443:0:(vvp_object.c:117:vvp_conf_set()) [0x200000401:0x9:0x0]: losing layout lock 00010000:00010000:0.0:1789761061.402948:0:5443:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff93f851b31000 lock: ffff93f8517ba000/0x4e3cc7995dd80f17 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x721975f94672feca expref: -99 pid: 11706 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1789761061.402957:0:5443:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff93f851b31000 lock: ffff93f8517bb000/0x4e3cc7995dd80f02 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x721975f94672fe68 expref: -99 pid: 11699 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1789761061.403002:0:5443:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff93f851b31000 lock: ffff93f8517bb400/0x4e3cc7995dd80f1e lrc: 1/0,0 mode: --/PR res: [0x200000401:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x721975f94672fee6 expref: -99 pid: 11706 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1789761061.403008:0:5443:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff93f851b31000 lock: ffff93f8517ba000/0x4e3cc7995dd80f17 lrc: 1/0,0 mode: --/PR res: [0x200000401:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x721975f94672feca expref: -99 pid: 11706 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1789761061.403012:0:5443:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff93f851b31000 lock: ffff93f8517bb000/0x4e3cc7995dd80f02 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x721975f94672fe68 expref: -99 pid: 11699 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1789761061.403016:0:5443:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:0.0:1789761061.403052:0:5443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff93f851b31000 lock: ffff93f8517bb400/0x4e3cc7995dd80f1e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x721975f94672fee6 expref: -99 pid: 11706 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1789761061.403074:0:5443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff93f851b31000 lock: ffff93f8517ba000/0x4e3cc7995dd80f17 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x721975f94672feca expref: -99 pid: 11706 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1789761061.403083:0:5443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff93f851b31000 lock: ffff93f8517bb000/0x4e3cc7995dd80f02 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x721975f94672fe68 expref: -99 pid: 11699 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1789761061.403093:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761061.403095:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761061.403097:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761061.403097:0:2357:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@ffff93f850d42a00 x1876699565635840/t0(0) o103->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:17/18 lens 336/224 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761062.362793:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761062.362803:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761062.362809:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761062.362822:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 997 00000100:00080000:3.0:1789761062.362834:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761062.362845:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761062.362896:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565635968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761062.362926:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761062.362931:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761062.362934:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761062.362940:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 997 00000100:00080000:3.0:1789761062.362944:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761062.362950:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761062.362960:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565635968:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789761062.362961:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761062.362981:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565636096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761062.362994:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761062.363000:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761062.363002:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761062.363008:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 997 00000100:00080000:3.0:1789761062.363013:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761062.363017:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761062.363024:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761062.363039:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565636224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761062.363042:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761062.363086:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761062/real 1789761062] req@ffff93f844d67480 x1876699565635968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761062.363102:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565635968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761062.363115:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761062.363120:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761062.363126:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565635968:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761062.363134:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761062.363138:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761062.363140:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761062.363145:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 997 00000100:00080000:3.0:1789761062.363150:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761062.363153:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761062.363158:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761062.363163:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565636096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761062.363172:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565636352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761062.363191:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761062.363204:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761062/real 1789761062] req@ffff93f844d67800 x1876699565636096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761062.363214:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565636096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761062.363220:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761062.363223:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761062.363225:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565636096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761062.363230:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565636224:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761062.363243:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761062.363250:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761062/real 1789761062] req@ffff93f844d67100 x1876699565636224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761062.363258:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565636224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761062.363263:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761062.363265:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761062.363268:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565636224:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761062.363279:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565636352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761062.363291:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761062.363297:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761062/real 1789761062] req@ffff93f844d66a00 x1876699565636352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761062.363310:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565636352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761062.363317:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761062.363318:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761062.363320:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565636352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761062.426768:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761063.450795:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761064.474780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761065.498823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761066.522796:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761067.482758:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761067.482773:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761067.482779:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761067.482794:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1002 00000100:00080000:2.0:1789761067.482807:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761067.482817:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761067.482869:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565636480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761067.482906:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761067.482911:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761067.482913:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761067.482918:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1002 00000100:00080000:2.0:1789761067.482923:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761067.482931:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761067.482942:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761067.482962:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565636480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761067.482963:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565636608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761067.482982:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761067.482986:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761067.482988:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761067.482993:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1002 00000100:00080000:2.0:1789761067.482998:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761067.483002:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761067.483009:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761067.483023:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565636736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761067.483036:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761067.483040:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761067.483042:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761067.483047:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1002 00000800:00000100:1.0:1789761067.483053:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761067.483054:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761067.483062:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761067.483069:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761067.483083:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565636864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761067.483106:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761067/real 1789761067] req@ffff93f8517c0000 x1876699565636480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761067.483126:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565636480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761067.483150:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761067.483155:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761067.483160:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565636480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761067.483190:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565636608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761067.483212:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761067.483222:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761067/real 1789761067] req@ffff93f8517c3b80 x1876699565636608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761067.483231:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565636608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761067.483242:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761067.483245:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761067.483247:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565636608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761067.483251:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565636736:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761067.483268:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761067.483280:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761067/real 1789761067] req@ffff93f8517c1f80 x1876699565636736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761067.483305:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565636736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761067.483315:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761067.483319:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761067.483323:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565636736:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761067.483330:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565636864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761067.483367:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761067.483378:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761067/real 1789761067] req@ffff93f8517c2300 x1876699565636864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761067.483386:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565636864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761067.483392:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761067.483394:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761067.483397:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565636864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761067.546804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761068.570749:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761069.594788:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761070.618820:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761071.642813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761072.602782:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761072.602795:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761072.602800:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761072.602812:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1007 00000100:00080000:2.0:1789761072.602823:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761072.602829:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761072.602867:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565636992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761072.602902:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761072.602906:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761072.602908:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761072.602913:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1007 00000100:00080000:2.0:1789761072.602917:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761072.602923:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761072.602936:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761072.602952:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565637120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761072.602961:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1789761072.602971:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565636992:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761072.602971:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761072.602976:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761072.602993:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1007 00000100:00080000:2.0:1789761072.602997:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761072.603000:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761072.603004:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761072.603022:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565637248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761072.603029:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761072.603032:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761072.603033:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761072.603036:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1007 00000100:00080000:2.0:1789761072.603045:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761072.603048:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761072.603051:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761072.603058:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565637376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761072.603074:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761072.603125:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761072/real 1789761072] req@ffff93f8517c2680 x1876699565636992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761072.603140:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565636992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761072.603153:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761072.603158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761072.603163:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565636992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761072.603195:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565637120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761072.603219:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761072.603229:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761072/real 1789761072] req@ffff93f8517c0380 x1876699565637120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761072.603238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565637120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761072.603244:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761072.603247:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761072.603249:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565637120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761072.603254:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565637248:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761072.603267:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761072.603274:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761072/real 1789761072] req@ffff93f8517c1500 x1876699565637248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761072.603282:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565637248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761072.603288:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761072.603289:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761072.603291:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565637248:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761072.603295:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565637376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761072.603307:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761072.603334:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761072/real 1789761072] req@ffff93f8517c0a80 x1876699565637376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761072.603343:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565637376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761072.603348:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761072.603351:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761072.603353:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565637376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761072.666791:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761073.690813:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761074.714783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761075.738790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761076.762803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761077.722754:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761077.722762:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761077.722766:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761077.722777:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1012 00000100:00080000:2.0:1789761077.722787:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761077.722793:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761077.722826:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565637504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761077.722855:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761077.722858:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761077.722860:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761077.722863:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1012 00000100:00080000:2.0:1789761077.722865:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761077.722868:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761077.722880:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761077.722895:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565637632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761077.722908:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761077.722912:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761077.722915:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789761077.722920:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565637504:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761077.722924:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1012 00000100:00080000:2.0:1789761077.722934:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761077.722940:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761077.722948:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761077.722965:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565637760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761077.722979:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761077.722983:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761077.722985:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761077.722989:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1012 00000100:00080000:2.0:1789761077.722995:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761077.723000:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789761077.723007:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1789761077.723009:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761077.723029:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565637888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761077.723062:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761077/real 1789761077] req@ffff93f8517c1180 x1876699565637504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761077.723079:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565637504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761077.723088:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761077.723092:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761077.723095:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565637504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761077.723119:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565637632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761077.723135:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761077.723144:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761077/real 1789761077] req@ffff93f8517c3100 x1876699565637632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761077.723152:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565637632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761077.723158:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761077.723161:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761077.723163:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565637632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761077.723167:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565637760:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761077.723178:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761077.723184:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761077/real 1789761077] req@ffff93f8517c2a00 x1876699565637760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761077.723196:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565637760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761077.723202:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761077.723203:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761077.723205:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565637760:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761077.723209:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565637888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761077.723219:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761077.723225:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761077/real 1789761077] req@ffff93f8517c2d80 x1876699565637888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761077.723231:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565637888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761077.723236:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761077.723239:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761077.723241:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565637888:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761077.786839:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761078.810916:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761079.834786:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761080.858812:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761081.882795:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761082.842717:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761082.842725:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761082.842729:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761082.842737:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1017 00000100:00080000:2.0:1789761082.842744:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761082.842750:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761082.842781:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565638016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761082.842803:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761082.842805:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761082.842807:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761082.842810:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1017 00000100:00080000:2.0:1789761082.842812:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761082.842814:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761082.842822:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761082.842831:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565638144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761082.842838:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761082.842840:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761082.842841:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761082.842843:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1017 00000100:00080000:2.0:1789761082.842846:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789761082.842850:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565638016:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761082.842851:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761082.842858:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761082.842865:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565638272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761082.842880:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761082.842884:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761082.842886:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761082.842890:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1017 00000100:00080000:2.0:1789761082.842894:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761082.842899:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761082.842905:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789761082.842909:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761082.842920:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565638400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761082.842946:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761082/real 1789761082] req@ffff93f8517c1880 x1876699565638016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761082.842959:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565638016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761082.842967:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761082.842970:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761082.842973:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565638016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761082.842978:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565638144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761082.843009:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761082.843019:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761082/real 1789761082] req@ffff93f8517c3480 x1876699565638144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761082.843028:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565638144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761082.843034:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761082.843037:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761082.843039:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565638144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761082.843059:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565638272:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761082.843075:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761082.843083:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761082/real 1789761082] req@ffff93f8517c3800 x1876699565638272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761082.843091:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565638272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761082.843098:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761082.843100:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761082.843102:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565638272:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761082.843107:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565638400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761082.843118:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761082.843146:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761082/real 1789761082] req@ffff93f8517c0e00 x1876699565638400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761082.843154:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565638400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761082.843159:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761082.843160:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761082.843162:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565638400:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761082.906809:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761083.930807:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761084.954824:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761085.978840:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761087.002798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761087.962720:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761087.962728:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761087.962732:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761087.962742:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1022 00000100:00080000:2.0:1789761087.962754:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761087.962760:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761087.962797:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565638528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761087.962819:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761087.962821:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761087.962823:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761087.962826:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1022 00000100:00080000:2.0:1789761087.962828:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761087.962830:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761087.962844:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761087.962856:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565638656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761087.962864:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761087.962868:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761087.962870:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789761087.962871:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565638528:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761087.962874:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1022 00000100:00080000:2.0:1789761087.962879:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761087.962883:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761087.962886:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761087.962894:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565638784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761087.962901:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761087.962903:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761087.962905:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761087.962907:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1022 00000100:00080000:2.0:1789761087.962911:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761087.962913:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761087.962916:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761087.962923:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565638912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761087.962934:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761087.962969:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761087/real 1789761087] req@ffff93f8517c0e00 x1876699565638528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761087.962983:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565638528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761087.962990:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761087.962993:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761087.962996:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565638528:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761087.963022:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565638656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761087.963042:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761087.963051:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761087/real 1789761087] req@ffff93f8517c3800 x1876699565638656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761087.963059:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565638656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761087.963065:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761087.963067:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761087.963069:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565638656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761087.963072:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565638784:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761087.963086:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761087.963096:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761087/real 1789761087] req@ffff93f8517c3480 x1876699565638784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761087.963109:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565638784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761087.963117:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761087.963120:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761087.963124:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565638784:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761087.963129:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565638912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761087.963151:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761087.963164:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761087/real 1789761087] req@ffff93f8517c1880 x1876699565638912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761087.963184:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565638912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761087.963203:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761087.963207:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761087.963210:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565638912:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761088.026798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761089.050762:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761090.074770:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761091.098810:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761092.122827:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761093.082791:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761093.082803:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761093.082807:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761093.082830:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1028 00000100:00080000:3.0:1789761093.082840:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761093.082847:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761093.082889:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565639040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761093.082915:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761093.082918:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761093.082920:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761093.082924:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1028 00000100:00080000:3.0:1789761093.082926:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761093.082929:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761093.082938:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761093.082949:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565639040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761093.082950:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565639168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761093.082967:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761093.082970:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761093.082972:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761093.082975:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1028 00000100:00080000:3.0:1789761093.082976:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761093.082979:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761093.082982:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761093.082991:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565639296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761093.083002:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761093.083005:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761093.083007:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761093.083012:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1028 00000100:00080000:3.0:1789761093.083016:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761093.083020:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761093.083026:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789761093.083036:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761093.083041:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565639424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761093.083086:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761093/real 1789761093] req@ffff93f844d65c00 x1876699565639040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761093.083105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565639040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761093.083117:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761093.083122:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761093.083127:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565639040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761093.083168:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565639168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761093.083192:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761093.083207:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761093/real 1789761093] req@ffff93f844d64000 x1876699565639168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761093.083221:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565639168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761093.083232:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761093.083236:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761093.083240:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565639168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761093.083245:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565639296:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761093.083263:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761093.083272:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761093/real 1789761093] req@ffff93f844d64e00 x1876699565639296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761093.083280:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565639296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761093.083286:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761093.083289:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761093.083291:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565639296:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761093.083294:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565639424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761093.083307:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761093.083313:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761093/real 1789761093] req@ffff93f844d65500 x1876699565639424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761093.083320:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565639424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761093.083327:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761093.083329:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761093.083333:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565639424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761093.146807:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761094.170788:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761095.194763:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761096.218813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761097.242814:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761098.203217:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761098.203226:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761098.203232:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761098.203242:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1033 00000100:00080000:2.0:1789761098.203251:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761098.203259:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761098.203302:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565639552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761098.203326:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761098.203329:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761098.203331:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761098.203333:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1033 00000100:00080000:2.0:1789761098.203336:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761098.203338:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761098.203347:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761098.203359:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565639680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761098.203369:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761098.203372:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761098.203374:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761098.203378:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1033 00000100:00080000:2.0:1789761098.203382:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761098.203386:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761098.203391:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761098.203402:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565639808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761098.203415:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761098.203418:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761098.203420:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761098.203422:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1033 00000100:00080000:2.0:1789761098.203425:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761098.203427:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761098.203432:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761098.203448:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565639936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761098.203464:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565639552:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1789761098.203534:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761098.203574:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761098/real 1789761098] req@ffff93f8517c2d80 x1876699565639552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761098.203590:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565639552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761098.203598:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761098.203601:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761098.203605:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565639552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761098.203618:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565639680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761098.203686:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761098.203716:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761098/real 1789761098] req@ffff93f8517c2a00 x1876699565639680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761098.203731:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565639680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761098.203741:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761098.203745:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761098.203749:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565639680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761098.203757:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565639808:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761098.203790:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761098.203806:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761098/real 1789761098] req@ffff93f8517c3100 x1876699565639808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761098.203819:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565639808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761098.203827:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761098.203831:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761098.203834:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565639808:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761098.203858:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565639936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761098.203875:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761098.203885:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761098/real 1789761098] req@ffff93f8517c1180 x1876699565639936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761098.203896:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565639936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761098.203905:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761098.203908:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761098.203911:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565639936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761098.266749:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761099.290782:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761100.314811:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761101.338774:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761102.362820:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761103.322735:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761103.322744:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761103.322747:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761103.322758:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1038 00000100:00080000:2.0:1789761103.322768:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761103.322775:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761103.322811:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565640064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761103.322833:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761103.322836:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761103.322837:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761103.322840:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1038 00000100:00080000:2.0:1789761103.322842:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761103.322845:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761103.322853:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761103.322862:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565640192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761103.322870:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761103.322873:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761103.322874:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761103.322876:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1038 00000100:00080000:2.0:1789761103.322878:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761103.322883:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761103.322884:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565640064:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789761103.322886:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761103.322897:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565640320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761103.322904:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761103.322906:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761103.322907:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761103.322910:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1038 00000100:00080000:2.0:1789761103.322913:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761103.322916:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761103.322918:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761103.322927:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565640448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761103.322945:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761103.322981:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761103/real 1789761103] req@ffff93f8517c0a80 x1876699565640064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761103.322994:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565640064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761103.323009:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761103.323012:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761103.323014:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565640064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761103.323020:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565640192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761103.323035:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761103.323044:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761103/real 1789761103] req@ffff93f8517c1500 x1876699565640192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761103.323052:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565640192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761103.323057:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761103.323059:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761103.323061:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565640192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761103.323094:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565640320:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761103.323114:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761103.323123:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761103/real 1789761103] req@ffff93f8517c0380 x1876699565640320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761103.323138:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565640320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761103.323148:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761103.323150:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761103.323152:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565640320:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761103.323156:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565640448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761103.323169:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761103.323176:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761103/real 1789761103] req@ffff93f8517c2680 x1876699565640448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761103.323183:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565640448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761103.323188:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761103.323190:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761103.323192:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565640448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761103.386785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761104.410816:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761105.434821:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761106.458867:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761107.482790:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761108.442780:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761108.442793:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761108.442798:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761108.442810:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1043 00000100:00080000:3.0:1789761108.442823:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761108.442831:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761108.442873:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565640576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761108.442902:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761108.442905:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761108.442907:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761108.442911:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1043 00000100:00080000:3.0:1789761108.442915:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761108.442919:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761108.442929:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761108.442941:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565640576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761108.442945:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565640704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761108.442959:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761108.442962:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761108.442964:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761108.442968:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1043 00000100:00080000:3.0:1789761108.442972:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761108.442975:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761108.442980:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761108.442993:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565640832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761108.443003:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761108.443017:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761108.443019:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789761108.443026:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761108.443027:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1043 00000100:00080000:3.0:1789761108.443037:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761108.443055:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761108.443061:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761108.443075:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565640960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761108.443102:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761108/real 1789761108] req@ffff93f844d65f80 x1876699565640576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761108.443125:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565640576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761108.443137:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761108.443152:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761108.443160:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565640576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761108.443192:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565640704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761108.443217:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761108.443233:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761108/real 1789761108] req@ffff93f844d66d80 x1876699565640704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761108.443248:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565640704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761108.443258:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761108.443261:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761108.443265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565640704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761108.443272:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565640832:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761108.443296:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761108.443309:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761108/real 1789761108] req@ffff93f844d64a80 x1876699565640832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761108.443323:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565640832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761108.443332:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761108.443336:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761108.443339:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565640832:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761108.443346:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565640960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761108.443366:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761108.443377:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761108/real 1789761108] req@ffff93f844d64380 x1876699565640960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761108.443390:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565640960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761108.443399:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761108.443403:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761108.443406:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565640960:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761108.506823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761109.530755:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761110.554822:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761111.578785:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761112.602873:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761113.562722:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761113.562730:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761113.562734:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761113.562742:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1048 00000100:00080000:3.0:1789761113.562750:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761113.562756:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761113.562789:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565641088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761113.562812:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761113.562815:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761113.562817:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761113.562820:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1048 00000100:00080000:3.0:1789761113.562822:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761113.562824:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761113.562833:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761113.562843:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565641216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761113.562851:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761113.562854:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761113.562855:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761113.562858:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1048 00000100:00080000:3.0:1789761113.562859:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761113.562862:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761113.562866:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761113.562868:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565641088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761113.562875:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565641344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761113.562884:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761113.562887:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761113.562888:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761113.562891:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1048 00000100:00080000:3.0:1789761113.562893:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761113.562895:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761113.562898:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761113.562905:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565641472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761113.562935:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761113.562978:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761113/real 1789761113] req@ffff93f844d65180 x1876699565641088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761113.562996:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565641088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761113.563024:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761113.563029:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761113.563033:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565641088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761113.563041:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565641216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761113.563065:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761113.563081:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761113/real 1789761113] req@ffff93f844d66300 x1876699565641216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761113.563103:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565641216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761113.563112:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761113.563115:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761113.563119:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565641216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761113.563147:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565641344:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761113.563171:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761113.563185:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761113/real 1789761113] req@ffff93f844d64380 x1876699565641344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761113.563197:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565641344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761113.563206:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761113.563210:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761113.563214:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565641344:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761113.563220:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565641472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761113.563238:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761113.563249:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761113/real 1789761113] req@ffff93f844d64a80 x1876699565641472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761113.563260:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565641472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761113.563275:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761113.563279:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761113.563282:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565641472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761113.626754:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761114.650823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761115.674786:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761116.698825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761117.722886:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761118.682715:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761118.682725:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761118.682730:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761118.682741:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1053 00000100:00080000:3.0:1789761118.682758:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761118.682766:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761118.682807:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565641600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761118.682831:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761118.682834:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761118.682836:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761118.682839:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1053 00000100:00080000:3.0:1789761118.682842:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761118.682846:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761118.682855:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761118.682860:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565641600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761118.682868:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565641728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761118.682876:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761118.682879:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761118.682880:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761118.682883:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1053 00000100:00080000:3.0:1789761118.682885:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761118.682888:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761118.682892:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761118.682902:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565641856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761118.682910:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761118.682912:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761118.682914:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761118.682917:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1053 00000100:00080000:3.0:1789761118.682920:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761118.682923:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761118.682925:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761118.682933:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565641984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761118.682997:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565641728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761118.683033:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565641856:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761118.683050:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565641984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761118.746815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761119.770876:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761120.794786:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1789761121.754962:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1789761121.755011:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1789761121.755029:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789761121.755076:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789761121.755084:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789761121.755091:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789761121.755130:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761118/real 1789761121] req@ffff93f844d65f80 x1876699565641728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761121.755160:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565641728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761121.755175:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761121.755182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761121.755186:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565641728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761121.755197:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761118/real 1789761121] req@ffff93f844d65500 x1876699565641856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761121.755211:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565641856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761121.755221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761121.755225:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761121.755229:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565641856:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761121.755237:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761118/real 1789761121] req@ffff93f844d64e00 x1876699565641984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761121.755249:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565641984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761121.755258:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761121.755262:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761121.755265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565641984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761121.755279:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761118/real 1789761121] req@ffff93f844d66d80 x1876699565641600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761121.755293:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565641600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761121.755300:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761121.755304:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761121.755307:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565641600:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1789761121.818865:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761122.842848:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761123.802832:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761123.802842:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761123.802847:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761123.802858:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1058 00000100:00080000:2.0:1789761123.802868:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761123.802875:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761123.802917:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565642112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761123.802969:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761123.802974:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761123.802976:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761123.802981:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1058 00000100:00080000:2.0:1789761123.802986:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761123.802991:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761123.803003:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761123.803020:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565642112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761123.803029:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565642240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761123.803043:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761123.803047:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761123.803049:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761123.803055:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1058 00000100:00080000:2.0:1789761123.803060:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761123.803063:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761123.803069:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761123.803080:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565642368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761123.803088:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761123.803091:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761123.803093:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761123.803096:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1058 00000100:00080000:2.0:1789761123.803099:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761123.803104:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761123.803109:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789761123.803110:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761123.803126:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565642496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761123.803173:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761123/real 1789761123] req@ffff93f8517c2300 x1876699565642112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761123.803195:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565642112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761123.803207:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761123.803212:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761123.803216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565642112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761123.803252:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565642240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761123.803279:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761123.803294:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761123/real 1789761123] req@ffff93f8517c1f80 x1876699565642240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761123.803307:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565642240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761123.803315:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761123.803319:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761123.803322:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565642240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761123.803328:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565642368:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761123.803349:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761123.803360:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761123/real 1789761123] req@ffff93f8517c3b80 x1876699565642368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761123.803371:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565642368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761123.803385:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761123.803389:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761123.803392:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565642368:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761123.803398:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565642496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761123.803418:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761123.803430:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761123/real 1789761123] req@ffff93f8517c0000 x1876699565642496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761123.803440:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565642496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761123.803447:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761123.803451:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761123.803453:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565642496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761123.866842:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761124.890877:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761125.914839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761126.938842:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761127.962986:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761128.922752:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761128.922762:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761128.922767:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761128.922778:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1063 00000100:00080000:3.0:1789761128.922788:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761128.922794:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761128.922838:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565642624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761128.922860:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761128.922863:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761128.922865:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761128.922867:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1063 00000100:00080000:3.0:1789761128.922870:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761128.922873:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761128.922881:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761128.922904:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565642752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761128.922911:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761128.922914:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761128.922915:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761128.922918:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1063 00000100:00080000:3.0:1789761128.922922:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761128.922924:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761128.922928:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761128.922936:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565642880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761128.922945:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565642624:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761128.922947:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761128.922950:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761128.922951:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761128.922954:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1063 00000100:00080000:3.0:1789761128.922956:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761128.922958:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761128.922960:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761128.922987:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565643008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761128.923043:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761128.923079:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761128/real 1789761128] req@ffff93f844d64000 x1876699565642624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761128.923152:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565642624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761128.923165:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761128.923169:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761128.923174:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565642624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761128.923183:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565642752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761128.923211:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761128.923226:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761128/real 1789761128] req@ffff93f844d65c00 x1876699565642752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761128.923238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565642752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761128.923245:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761128.923247:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761128.923249:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565642752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761128.923272:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565642880:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761128.923288:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761128.923296:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761128/real 1789761128] req@ffff93f844d66a00 x1876699565642880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761128.923304:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565642880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761128.923310:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761128.923312:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761128.923314:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565642880:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761128.923318:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565643008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761128.923328:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761128.923350:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761128/real 1789761128] req@ffff93f844d67100 x1876699565643008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761128.923359:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565643008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761128.923365:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761128.923367:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761128.923369:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565643008:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761128.986770:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761130.010829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761131.034790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761132.058798:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761133.082767:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761134.042776:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761134.042792:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761134.042796:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761134.042809:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1069 00000100:00080000:3.0:1789761134.042822:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761134.042833:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761134.042888:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565643136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761134.042917:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761134.042932:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761134.042935:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761134.042948:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1069 00000100:00080000:3.0:1789761134.042952:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761134.042958:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761134.042959:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565643136:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789761134.042967:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761134.042981:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565643264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761134.042989:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761134.042992:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761134.042993:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761134.042996:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1069 00000100:00080000:3.0:1789761134.042999:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761134.043002:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761134.043006:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761134.043015:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565643392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761134.043023:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761134.043025:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761134.043026:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761134.043028:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1069 00000100:00080000:3.0:1789761134.043031:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789761134.043033:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761134.043037:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761134.043041:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761134.043047:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565643520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761134.043097:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761134/real 1789761134] req@ffff93f844d67800 x1876699565643136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761134.043112:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565643136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761134.043120:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761134.043123:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761134.043127:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565643136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761134.043148:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565643264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761134.043170:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761134.043179:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761134/real 1789761134] req@ffff93f844d67480 x1876699565643264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761134.043193:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565643264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761134.043199:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761134.043201:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761134.043203:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565643264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761134.043208:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565643392:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761134.043227:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761134.043238:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761134/real 1789761134] req@ffff93f844d65880 x1876699565643392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761134.043246:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565643392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761134.043252:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761134.043254:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761134.043256:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565643392:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761134.043259:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565643520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761134.043278:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761134.043290:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761134/real 1789761134] req@ffff93f844d66680 x1876699565643520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761134.043306:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565643520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761134.043331:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761134.043336:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761134.043339:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565643520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761134.106825:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761135.130822:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761136.154821:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761137.178886:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761138.202790:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761139.162764:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761139.162776:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761139.162780:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761139.162792:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1074 00000100:00080000:3.0:1789761139.162804:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761139.162825:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761139.162868:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565643648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761139.162897:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761139.162900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761139.162901:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761139.162904:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1074 00000100:00080000:3.0:1789761139.162907:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761139.162910:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761139.162918:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761139.162927:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565643776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761139.162935:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761139.162938:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761139.162939:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761139.162942:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1074 00000100:00080000:3.0:1789761139.162944:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761139.162946:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761139.162949:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761139.162957:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565643648:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761139.162961:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565643904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761139.162972:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761139.162974:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761139.162975:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761139.162987:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1074 00000100:00080000:3.0:1789761139.162993:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761139.162996:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761139.163000:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761139.163008:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565644032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761139.163044:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761139.163094:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761139/real 1789761139] req@ffff93f844d64700 x1876699565643648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761139.163110:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565643648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761139.163120:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761139.163123:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761139.163126:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565643648:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761139.163131:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565643776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761139.163150:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761139.163159:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761139/real 1789761139] req@ffff93f844d67b80 x1876699565643776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761139.163167:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565643776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761139.163172:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761139.163175:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761139.163177:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565643776:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761139.163205:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565643904:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761139.163217:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761139.163224:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761139/real 1789761139] req@ffff93f844d66680 x1876699565643904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761139.163232:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565643904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761139.163237:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761139.163239:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761139.163241:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565643904:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761139.163244:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565644032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761139.163255:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761139.163261:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761139/real 1789761139] req@ffff93f844d65880 x1876699565644032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761139.163268:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565644032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761139.163273:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761139.163275:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761139.163277:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565644032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761139.226828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761140.250793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761141.274790:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761142.298831:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761143.322833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761144.282738:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761144.282750:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761144.282755:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761144.282765:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1079 00000100:00080000:3.0:1789761144.282788:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761144.282796:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761144.282834:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565644160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761144.282861:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761144.282865:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761144.282867:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761144.282871:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1079 00000100:00080000:3.0:1789761144.282875:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761144.282880:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761144.282890:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761144.282899:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565644160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761144.282906:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565644288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761144.282924:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761144.282928:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761144.282930:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761144.282934:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1079 00000100:00080000:3.0:1789761144.282939:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761144.282942:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761144.282947:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789761144.282958:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761144.282962:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565644416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761144.282973:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761144.282977:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761144.282979:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761144.282984:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1079 00000100:00080000:3.0:1789761144.282988:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761144.282992:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761144.282997:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761144.283007:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761144/real 1789761144] req@ffff93f844d67480 x1876699565644160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761144.283011:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565644544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761144.283035:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565644160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761144.283044:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761144.283048:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761144.283052:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565644160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761144.283071:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565644288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761144.283087:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761144.283096:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761144/real 1789761144] req@ffff93f844d67800 x1876699565644288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761144.283105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565644288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761144.283116:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761144.283119:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761144.283124:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565644288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761144.283128:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565644416:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761144.283141:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761144.283148:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761144/real 1789761144] req@ffff93f844d67100 x1876699565644416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761144.283161:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565644416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761144.283167:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761144.283169:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761144.283171:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565644416:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761144.283174:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565644544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761144.283186:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761144.283195:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761144/real 1789761144] req@ffff93f844d66a00 x1876699565644544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761144.283207:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565644544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761144.283216:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761144.283219:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761144.283223:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565644544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761144.346815:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761145.370825:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761146.394769:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761147.418780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761148.442806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761149.402873:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761149.402884:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761149.402889:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761149.402903:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1084 00000100:00080000:2.0:1789761149.402913:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761149.402920:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761149.402961:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565644672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761149.402987:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761149.402992:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761149.402994:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761149.402998:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1084 00000100:00080000:2.0:1789761149.403003:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761149.403008:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761149.403018:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761149.403025:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565644672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761149.403039:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565644800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761149.403052:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761149.403056:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761149.403059:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761149.403063:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1084 00000100:00080000:2.0:1789761149.403068:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761149.403071:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761149.403076:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761149.403089:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565644928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761149.403098:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761149.403107:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761149.403121:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761149.403124:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761149.403129:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1084 00000100:00080000:2.0:1789761149.403134:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761149.403139:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761149.403145:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761149/real 1789761149] req@ffff93f8517c0000 x1876699565644672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1789761149.403148:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761149.403167:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565645056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761149.403171:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565644672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761149.403199:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761149.403205:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761149.403210:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565644672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761149.403233:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565644800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761149.403256:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761149.403269:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761149/real 1789761149] req@ffff93f8517c3b80 x1876699565644800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761149.403281:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565644800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761149.403289:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761149.403293:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761149.403296:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565644800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761149.403302:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565644928:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761149.403330:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761149.403341:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761149/real 1789761149] req@ffff93f8517c1f80 x1876699565644928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761149.403353:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565644928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761149.403362:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761149.403365:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761149.403368:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565644928:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761149.403379:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565645056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761149.403397:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761149.403407:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761149/real 1789761149] req@ffff93f8517c2300 x1876699565645056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761149.403419:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565645056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761149.403427:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761149.403430:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761149.403433:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565645056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761149.466822:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761150.490786:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761151.514836:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761152.538797:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761153.562824:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761154.522758:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761154.522766:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761154.522770:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761154.522778:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1089 00000100:00080000:3.0:1789761154.522785:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761154.522791:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761154.522823:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565645184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761154.522845:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761154.522847:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761154.522849:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761154.522851:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1089 00000100:00080000:3.0:1789761154.522854:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761154.522856:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761154.522864:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761154.522873:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565645312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761154.522881:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761154.522884:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761154.522885:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761154.522890:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1089 00000100:00100000:0.0:1789761154.522892:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565645184:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761154.522893:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761154.522899:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761154.522905:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761154.522913:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565645440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761154.522925:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761154.522927:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761154.522929:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761154.522931:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1089 00000100:00080000:3.0:1789761154.522934:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761154.522936:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761154.522939:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761154.522946:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565645568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761154.522956:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761154.522991:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761154/real 1789761154] req@ffff93f844d65c00 x1876699565645184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761154.523004:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565645184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761154.523011:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761154.523014:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761154.523018:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565645184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761154.523037:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565645312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761154.523054:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761154.523064:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761154/real 1789761154] req@ffff93f844d64000 x1876699565645312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761154.523078:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565645312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761154.523086:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761154.523091:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761154.523094:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565645312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761154.523101:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565645440:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761154.523123:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761154.523131:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761154/real 1789761154] req@ffff93f844d66d80 x1876699565645440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761154.523140:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565645440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761154.523146:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761154.523148:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761154.523150:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565645440:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761154.523154:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565645568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761154.523171:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761154.523182:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761154/real 1789761154] req@ffff93f844d64e00 x1876699565645568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761154.523202:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565645568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761154.523211:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761154.523214:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761154.523216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565645568:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761154.586828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761155.610829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761156.634809:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761157.658849:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761158.682878:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761159.642746:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761159.642766:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761159.642771:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761159.642779:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1094 00000100:00080000:3.0:1789761159.642789:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761159.642797:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761159.642832:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565645696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761159.642864:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761159.642868:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761159.642870:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761159.642874:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1094 00000100:00080000:3.0:1789761159.642879:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761159.642883:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761159.642893:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761159.642907:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565645824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761159.642916:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789761159.642919:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565645696:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761159.642919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761159.642921:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761159.642925:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1094 00000100:00080000:3.0:1789761159.642928:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761159.642931:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761159.642935:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761159.642943:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565645952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761159.642952:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761159.642955:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761159.642956:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761159.642959:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1094 00000100:00080000:3.0:1789761159.642962:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761159.642965:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761159.642969:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789761159.642977:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761159.642977:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565646080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761159.643017:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761159/real 1789761159] req@ffff93f844d65500 x1876699565645696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761159.643033:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565645696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761159.643040:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761159.643043:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761159.643046:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565645696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761159.643065:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565645824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761159.643084:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761159.643092:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761159/real 1789761159] req@ffff93f844d65f80 x1876699565645824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761159.643100:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565645824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761159.643110:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761159.643112:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761159.643114:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565645824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761159.643118:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565645952:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761159.643128:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761159.643135:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761159/real 1789761159] req@ffff93f844d64a80 x1876699565645952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761159.643150:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565645952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761159.643156:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761159.643158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761159.643160:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565645952:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761159.643163:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565646080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761159.643175:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761159.643181:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761159/real 1789761159] req@ffff93f844d64380 x1876699565646080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761159.643189:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565646080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761159.643193:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761159.643195:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761159.643197:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565646080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761159.706834:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761160.730859:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761161.754809:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761162.778850:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761163.802774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761164.762950:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761164.762961:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761164.762966:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761164.762978:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1099 00000100:00080000:3.0:1789761164.762989:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761164.762997:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761164.763034:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565646208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761164.763058:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761164.763063:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761164.763065:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761164.763069:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1099 00000100:00080000:3.0:1789761164.763073:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761164.763079:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761164.763090:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761164.763098:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565646208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761164.763107:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565646336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761164.763121:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761164.763125:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761164.763128:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761164.763132:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1099 00000100:00080000:3.0:1789761164.763138:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761164.763142:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761164.763148:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789761164.763163:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761164.763165:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565646464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761164.763175:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761164.763178:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761164.763180:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761164.763183:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1099 00000100:00080000:3.0:1789761164.763186:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761164.763189:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761164.763193:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761164.763203:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565646592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761164.763208:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761164/real 1789761164] req@ffff93f844d66300 x1876699565646208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761164.763244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565646208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761164.763256:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761164.763260:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761164.763265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565646208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761164.763289:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565646336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761164.763311:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761164.763323:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761164/real 1789761164] req@ffff93f844d65180 x1876699565646336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761164.763336:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565646336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761164.763344:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761164.763347:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761164.763350:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565646336:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761164.763356:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565646464:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761164.763375:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761164.763387:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761164/real 1789761164] req@ffff93f844d64380 x1876699565646464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761164.763398:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565646464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761164.763406:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761164.763409:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761164.763412:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565646464:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761164.763418:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565646592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761164.763434:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761164.763444:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761164/real 1789761164] req@ffff93f844d64a80 x1876699565646592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761164.763455:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565646592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761164.763463:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761164.763466:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761164.763469:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565646592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761164.826799:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761165.850783:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761166.874824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761167.898746:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761168.922762:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761169.882733:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761169.882741:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761169.882746:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761169.882755:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1104 00000100:00080000:0.0:1789761169.882766:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761169.882775:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761169.882815:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565646720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761169.882849:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761169.882853:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761169.882855:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761169.882858:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1104 00000100:00080000:0.0:1789761169.882863:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761169.882867:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761169.882876:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761169.882890:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565646848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761169.882895:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565646720:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761169.882904:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761169.882908:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761169.882909:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761169.882912:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1104 00000100:00080000:0.0:1789761169.882915:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761169.882919:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761169.882924:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761169.882938:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565646976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761169.882956:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1789761169.882961:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789761169.882964:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761169.882968:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761169.882974:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1104 00000100:00080000:0.0:1789761169.882977:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761169.882981:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761169.882986:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761169.882996:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565647104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761169.883057:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761169/real 1789761169] req@ffff93f850d40e00 x1876699565646720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761169.883072:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565646720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761169.883080:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761169.883083:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761169.883086:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565646720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761169.883104:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565646848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761169.883120:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761169.883129:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761169/real 1789761169] req@ffff93f850d41c00 x1876699565646848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761169.883137:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565646848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761169.883142:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761169.883144:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761169.883146:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565646848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761169.883150:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565646976:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761169.883161:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761169.883166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761169/real 1789761169] req@ffff93f850d42680 x1876699565646976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761169.883174:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565646976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761169.883179:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761169.883181:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761169.883183:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565646976:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761169.883186:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565647104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761169.883196:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761169.883202:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761169/real 1789761169] req@ffff93f850d40000 x1876699565647104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761169.883306:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565647104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761169.883312:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761169.883314:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761169.883317:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565647104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761169.946761:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761170.970769:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761171.994793:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761173.018777:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761174.042805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761175.002769:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761175.002780:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761175.002785:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761175.002795:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1109 00000100:00080000:3.0:1789761175.002806:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761175.002813:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761175.002855:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565647232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761175.002887:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761175.002890:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761175.002891:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761175.002894:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1109 00000100:00080000:3.0:1789761175.002897:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761175.002900:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761175.002908:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761175.002930:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565647232:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1789761175.003001:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761175.003052:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761175/real 1789761175] req@ffff93f844d65f80 x1876699565647232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761175.003074:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565647232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761175.003087:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761175.003092:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761175.003097:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565647232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761175.004148:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565647360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761175.004172:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761175.004175:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761175.004178:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761175.004182:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1109 00000100:00080000:3.0:1789761175.004192:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761175.004197:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761175.004204:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761175.004211:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565647360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761175.004215:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565647488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761175.004236:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761175.004239:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761175.004241:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761175.004248:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1109 00000100:00080000:3.0:1789761175.004252:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761175.004255:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789761175.004261:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1789761175.004264:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761175.004283:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761175/real 1789761175] req@ffff93f844d65500 x1876699565647360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761175.004289:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565647616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761175.004316:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565647360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761175.004329:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761175.004335:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761175.004341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565647360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761175.004360:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565647488:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761175.004380:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761175.004390:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761175/real 1789761175] req@ffff93f844d64e00 x1876699565647488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761175.004399:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565647488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761175.004405:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761175.004407:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761175.004409:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565647488:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761175.004413:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565647616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761175.004425:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761175.004432:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761175/real 1789761175] req@ffff93f844d66d80 x1876699565647616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761175.004440:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565647616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761175.004446:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761175.004449:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761175.004451:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565647616:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761175.066790:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761176.090819:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761177.114803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761178.138819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761179.162794:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761180.122780:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761180.122792:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761180.122796:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761180.122806:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1115 00000100:00080000:2.0:1789761180.122815:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761180.122823:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761180.122862:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565647744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761180.122890:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761180.122894:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761180.122896:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761180.122901:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1115 00000100:00080000:2.0:1789761180.122905:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761180.122909:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761180.122920:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761180.122943:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565647872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761180.122956:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761180.122962:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761180.122967:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789761180.122973:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565647744:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761180.122974:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1115 00000100:00080000:2.0:1789761180.122984:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761180.122991:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761180.122997:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761180.123011:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565648000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761180.123031:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761180.123036:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761180.123038:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761180.123046:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1115 00000100:00080000:2.0:1789761180.123051:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761180.123056:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761180.123061:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789761180.123080:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761180.123080:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565648128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761180.123150:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761180/real 1789761180] req@ffff93f8517c2680 x1876699565647744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761180.123173:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565647744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761180.123187:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761180.123191:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761180.123196:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565647744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761180.123236:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565647872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761180.123262:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761180.123276:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761180/real 1789761180] req@ffff93f8517c0380 x1876699565647872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761180.123289:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565647872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761180.123298:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761180.123303:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761180.123306:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565647872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761180.123313:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565648000:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761180.123330:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761180.123337:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761180/real 1789761180] req@ffff93f8517c1500 x1876699565648000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761180.123346:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565648000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761180.123352:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761180.123354:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761180.123356:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565648000:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761180.123360:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565648128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761180.123370:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761180.123376:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761180/real 1789761180] req@ffff93f8517c0a80 x1876699565648128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761180.123383:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565648128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761180.123389:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761180.123391:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761180.123393:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565648128:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761180.186815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761181.210799:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761182.234871:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761183.258801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761184.282811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761185.242752:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761185.242765:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761185.242770:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761185.242784:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1120 00000100:00080000:0.0:1789761185.242796:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761185.242805:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761185.242853:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565648256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761185.242900:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761185.242904:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761185.242907:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761185.242913:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1120 00000100:00080000:0.0:1789761185.242917:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761185.242923:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761185.242934:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761185.242946:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565648256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761185.242953:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565648384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761185.242967:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761185.242971:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761185.242980:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761185.242986:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1120 00000100:00080000:0.0:1789761185.242990:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761185.242995:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761185.243002:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761185.243017:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565648512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761185.243030:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761185.243033:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761185.243035:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761185.243039:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1120 00000100:00080000:0.0:1789761185.243044:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761185.243049:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761185.243066:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565648384:192.168.203.147@tcp:38:kworker.0 00000008:00100000:0.0:1789761185.243067:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761185.243089:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565648640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761185.243104:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565648512:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761185.243135:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565648640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761185.306813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761186.330801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761187.354764:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789761188.314921:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789761188.314968:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789761188.314990:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761188.315029:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761188.315039:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761188.315047:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1789761188.315104:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761185/real 1789761188] req@ffff93f850d41180 x1876699565648640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761188.315139:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565648640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761188.315151:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761188.315156:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761188.315161:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565648640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761188.315169:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761185/real 1789761188] req@ffff93f850d43100 x1876699565648384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761188.315182:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565648384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761188.315190:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761188.315194:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761188.315197:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565648384:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761188.315205:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761185/real 1789761188] req@ffff93f850d41f80 x1876699565648512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761188.315222:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565648512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761188.315230:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761188.315233:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761188.315236:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565648512:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761188.315242:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761185/real 1789761188] req@ffff93f850d41500 x1876699565648256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761188.315254:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565648256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761188.315261:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761188.315264:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761188.315268:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565648256:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761188.378805:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761189.402790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761190.362746:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761190.362758:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761190.362762:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761190.362773:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1125 00000100:00080000:2.0:1789761190.362783:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761190.362791:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761190.362830:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565648768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761190.362851:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761190.362857:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761190.362858:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761190.362861:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1125 00000100:00080000:2.0:1789761190.362865:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761190.362868:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761190.362877:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761190.362889:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565648896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761190.362891:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565648768:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761190.362904:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761190.362907:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761190.362908:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761190.362910:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1125 00000100:00080000:2.0:1789761190.362912:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761190.362915:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761190.362925:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761190.362938:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565649024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761190.362950:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761190.362954:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761190.362956:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761190.362961:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1125 00000100:00080000:2.0:1789761190.362967:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761190.362972:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761190.362978:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789761190.362990:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761190.362992:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565649152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761190.363048:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761190/real 1789761190] req@ffff93f8517c1180 x1876699565648768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761190.363070:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565648768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761190.363084:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761190.363089:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761190.363095:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565648768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761190.363128:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565648896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761190.363157:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761190.363172:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761190/real 1789761190] req@ffff93f8517c3100 x1876699565648896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761190.363187:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565648896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761190.363201:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761190.363205:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761190.363208:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565648896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761190.363216:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565649024:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761190.363240:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761190.363252:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761190/real 1789761190] req@ffff93f8517c2a00 x1876699565649024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761190.363265:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565649024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761190.363274:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761190.363277:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761190.363281:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565649024:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761190.363287:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565649152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761190.363309:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761190.363321:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761190/real 1789761190] req@ffff93f8517c2d80 x1876699565649152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761190.363334:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565649152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761190.363342:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761190.363346:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761190.363349:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565649152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761190.426812:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761191.450745:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761192.474776:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761193.498815:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761194.522776:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761195.482776:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761195.482790:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761195.482797:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761195.482810:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1130 00000100:00080000:3.0:1789761195.482823:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761195.482831:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761195.482877:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565649280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761195.482912:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761195.482917:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761195.482919:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761195.482924:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1130 00000100:00080000:3.0:1789761195.482930:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761195.482935:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761195.482947:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761195.482954:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565649280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761195.482967:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565649408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761195.482981:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761195.482985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761195.482987:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761195.482992:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1130 00000100:00080000:3.0:1789761195.482996:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761195.483001:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761195.483007:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761195.483020:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565649536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761195.483031:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761195.483035:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761195.483038:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789761195.483039:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761195.483045:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1130 00000100:00080000:3.0:1789761195.483049:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761195.483052:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761195.483057:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761195.483067:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565649664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761195.483096:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761195/real 1789761195] req@ffff93f844d64000 x1876699565649280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761195.483119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565649280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761195.483128:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761195.483132:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761195.483136:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565649280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761195.483162:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565649408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761195.483180:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761195.483192:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761195/real 1789761195] req@ffff93f844d65c00 x1876699565649408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761195.483205:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565649408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761195.483214:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761195.483218:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761195.483221:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565649408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761195.483228:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565649536:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761195.483251:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761195.483262:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761195/real 1789761195] req@ffff93f844d66a00 x1876699565649536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761195.483270:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565649536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761195.483276:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761195.483279:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761195.483281:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565649536:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761195.483284:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565649664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761195.483298:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761195.483305:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761195/real 1789761195] req@ffff93f844d67100 x1876699565649664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761195.483312:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565649664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761195.483318:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761195.483319:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761195.483321:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565649664:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761195.546838:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761196.570779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761197.594882:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761198.618803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761199.642818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761200.602770:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761200.602781:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761200.602786:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761200.602797:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1135 00000100:00080000:0.0:1789761200.602807:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761200.602814:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761200.602852:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565649792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761200.602881:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761200.602885:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761200.602887:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761200.602892:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1135 00000100:00080000:0.0:1789761200.602896:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761200.602901:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761200.602910:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761200.602920:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565649792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761200.602927:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565649920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761200.602940:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761200.602943:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761200.602945:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761200.602971:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1135 00000100:00080000:0.0:1789761200.602978:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761200.602982:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761200.602989:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789761200.602989:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761200.603007:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565650048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761200.603019:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761200.603023:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761200.603026:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761200.603033:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1135 00000100:00100000:3.0:1789761200.603035:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761200/real 1789761200] req@ffff93f850d43800 x1876699565649792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761200.603039:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761200.603047:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761200.603058:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761200.603066:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565649792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761200.603074:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565650176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761200.603083:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761200.603092:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761200.603097:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565649792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761200.603122:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565649920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761200.603144:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761200.603156:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761200/real 1789761200] req@ffff93f850d43480 x1876699565649920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761200.603168:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565649920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761200.603177:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761200.603180:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761200.603184:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565649920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761200.603189:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565650048:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761200.603207:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761200.603216:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761200/real 1789761200] req@ffff93f850d41500 x1876699565650048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761200.603227:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565650048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761200.603241:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761200.603244:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761200.603247:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565650048:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761200.603253:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565650176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761200.603269:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761200.603280:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761200/real 1789761200] req@ffff93f850d41f80 x1876699565650176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761200.603291:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565650176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761200.603298:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761200.603301:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761200.603305:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565650176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761200.666794:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761201.690805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761202.714787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761203.738756:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761204.762805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761205.722733:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761205.722743:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761205.722748:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761205.722759:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1140 00000100:00080000:0.0:1789761205.722770:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761205.722777:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761205.722819:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565650304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761205.722849:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761205.722852:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761205.722854:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761205.722857:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1140 00000100:00080000:0.0:1789761205.722860:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761205.722862:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761205.722871:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761205.722881:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565650432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761205.722889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761205.722893:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761205.722895:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1789761205.722895:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565650304:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761205.722899:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1140 00000100:00080000:0.0:1789761205.722902:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761205.722904:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761205.722911:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761205.722922:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565650560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761205.722935:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761205.722937:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761205.722939:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761205.722941:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1140 00000100:00080000:0.0:1789761205.722943:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761205.722946:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761205.722949:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761205.722957:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565650688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761205.722986:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761205.723049:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761205/real 1789761205] req@ffff93f850d43100 x1876699565650304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761205.723073:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565650304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761205.723087:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761205.723093:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761205.723098:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565650304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761205.723129:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565650432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761205.723160:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761205.723178:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761205/real 1789761205] req@ffff93f850d41180 x1876699565650432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761205.723194:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565650432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761205.723204:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761205.723209:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761205.723214:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565650432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761205.723290:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565650560:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761205.723334:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761205.723348:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761205/real 1789761205] req@ffff93f850d40000 x1876699565650560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761205.723361:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565650560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761205.723369:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761205.723373:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761205.723376:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565650560:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761205.723387:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565650688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761205.723544:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761205.723605:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761205/real 1789761205] req@ffff93f850d42680 x1876699565650688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761205.723620:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565650688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761205.723630:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761205.723634:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761205.723638:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565650688:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761205.786805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761206.810797:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761207.834773:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761208.858825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761209.882792:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761210.842773:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761210.842784:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761210.842790:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761210.842802:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1145 00000100:00080000:1.0:1789761210.842812:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761210.842835:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761210.842877:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565650816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761210.842903:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761210.842906:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761210.842908:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761210.842911:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1145 00000100:00080000:1.0:1789761210.842914:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761210.842917:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761210.842927:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761210.842936:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565650944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761210.842943:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565650816:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761210.842948:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761210.842953:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761210.842955:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761210.842959:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1145 00000100:00080000:1.0:1789761210.842961:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761210.842963:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761210.842966:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761210.842977:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565651072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761210.842984:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761210.842988:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761210.842989:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761210.842992:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1145 00000100:00080000:1.0:1789761210.842994:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761210.842996:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761210.842999:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761210.843006:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565651200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761210.843035:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761210.843090:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761210/real 1789761210] req@ffff93f844cc9f80 x1876699565650816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761210.843112:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565650816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761210.843142:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761210.843147:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761210.843151:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565650816:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761210.843204:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565650944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761210.843228:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761210.843241:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761210/real 1789761210] req@ffff93f844cc9880 x1876699565650944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761210.843256:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565650944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761210.843265:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761210.843269:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761210.843273:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565650944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761210.843291:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565651072:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761210.843313:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761210.843322:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761210/real 1789761210] req@ffff93f844cc9180 x1876699565651072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761210.843330:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565651072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761210.843336:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761210.843339:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761210.843341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565651072:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761210.843344:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565651200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761210.843357:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761210.843363:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761210/real 1789761210] req@ffff93f844cca300 x1876699565651200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761210.843370:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565651200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761210.843380:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761210.843383:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761210.843387:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565651200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761210.906831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761211.930792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761212.954801:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761213.978815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761215.002849:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761215.962739:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761215.962750:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761215.962756:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761215.962770:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1150 00000100:00080000:0.0:1789761215.962782:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761215.962791:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761215.962837:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565651328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761215.962863:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761215.962866:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761215.962868:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761215.962871:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1150 00000100:00080000:0.0:1789761215.962875:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761215.962879:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761215.962888:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761215.962901:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565651456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761215.962902:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565651328:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761215.962917:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761215.962920:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761215.962921:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761215.962924:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1150 00000100:00080000:0.0:1789761215.962927:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761215.962929:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761215.962933:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761215.962943:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565651584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761215.962950:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761215.962953:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761215.962955:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761215.962957:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1150 00000100:00080000:0.0:1789761215.962959:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761215.962962:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761215.962965:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761215.962973:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565651712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761215.962979:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761215.963036:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761215/real 1789761215] req@ffff93f850d41c00 x1876699565651328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761215.963057:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565651328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761215.963070:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761215.963073:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761215.963077:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565651328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761215.963103:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565651456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761215.963122:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761215.963134:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761215/real 1789761215] req@ffff93f850d40e00 x1876699565651456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761215.963160:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565651456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761215.963169:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761215.963173:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761215.963178:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565651456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761215.963185:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565651584:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761215.963211:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761215.963224:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761215/real 1789761215] req@ffff93f850d40700 x1876699565651584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761215.963249:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565651584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761215.963256:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761215.963258:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761215.963261:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565651584:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761215.963266:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565651712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761215.963284:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761215.963292:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761215/real 1789761215] req@ffff93f850d43b80 x1876699565651712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761215.963300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565651712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761215.963306:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761215.963308:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761215.963310:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565651712:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761216.026794:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761217.050879:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761218.074789:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761219.098829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761220.122784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761221.082715:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761221.082727:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761221.082732:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761221.082742:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1156 00000100:00080000:0.0:1789761221.082751:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761221.082759:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761221.082802:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565651840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761221.082822:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761221.082826:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761221.082828:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761221.082832:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1156 00000100:00080000:0.0:1789761221.082836:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761221.082841:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761221.082852:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761221.082855:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565651840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761221.082865:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565651968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761221.082874:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761221.082878:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761221.082879:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761221.082883:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1156 00000100:00080000:0.0:1789761221.082886:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761221.082889:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761221.082894:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761221.082904:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565652096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761221.082913:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761221.082916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:3.0:1789761221.082916:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789761221.082918:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761221.082922:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1156 00000100:00080000:0.0:1789761221.082925:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761221.082928:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761221.082932:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761221.082941:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565652224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761221.082959:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761221/real 1789761221] req@ffff93f850d40380 x1876699565651840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761221.082979:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565651840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761221.082990:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761221.082993:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761221.082996:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565651840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761221.083016:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565651968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761221.083035:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761221.083044:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761221/real 1789761221] req@ffff93f850d42300 x1876699565651968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761221.083053:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565651968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761221.083058:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761221.083060:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761221.083062:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565651968:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761221.083066:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565652096:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761221.083077:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761221.083083:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761221/real 1789761221] req@ffff93f850d43b80 x1876699565652096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761221.083095:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565652096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761221.083100:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761221.083102:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761221.083104:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565652096:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761221.083107:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565652224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761221.083125:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761221.083132:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761221/real 1789761221] req@ffff93f850d40700 x1876699565652224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761221.083139:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565652224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761221.083144:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761221.083146:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761221.083148:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565652224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761221.146772:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761222.170788:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761223.194777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761224.218821:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761225.242866:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761226.202786:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761226.202800:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761226.202806:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761226.202819:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1161 00000100:00080000:1.0:1789761226.202832:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761226.202851:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761226.202895:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565652352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761226.202920:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761226.202923:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761226.202925:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761226.202928:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1161 00000100:00080000:1.0:1789761226.202931:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761226.202935:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761226.202944:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761226.202964:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565652480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761226.202970:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565652352:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761226.202980:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761226.202986:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761226.202988:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761226.202993:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1161 00000100:00080000:1.0:1789761226.202998:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761226.203003:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761226.203008:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761226.203021:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565652608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761226.203030:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761226.203034:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761226.203035:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761226.203039:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1161 00000100:00080000:1.0:1789761226.203053:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761226.203059:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761226.203065:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789761226.203066:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761226.203081:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565652736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761226.203143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761226/real 1789761226] req@ffff93f844cc8380 x1876699565652352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761226.203180:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565652352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761226.203194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761226.203199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761226.203205:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565652352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761226.203232:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565652480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761226.203257:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761226.203273:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761226/real 1789761226] req@ffff93f844cc8a80 x1876699565652480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761226.203286:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565652480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761226.203296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761226.203300:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761226.203304:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565652480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761226.203311:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565652608:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761226.203333:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761226.203346:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761226/real 1789761226] req@ffff93f844ccbb80 x1876699565652608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761226.203358:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565652608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761226.203367:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761226.203370:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761226.203373:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565652608:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761226.203380:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565652736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761226.203402:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761226.203414:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761226/real 1789761226] req@ffff93f844cc8700 x1876699565652736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761226.203427:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565652736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761226.203435:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761226.203439:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761226.203442:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565652736:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761226.266849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761227.290776:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761228.314793:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761229.338803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761230.362776:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761231.322753:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761231.322769:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761231.322775:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761231.322788:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1166 00000100:00080000:1.0:1789761231.322801:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761231.322811:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761231.322858:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565652864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761231.322885:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761231.322889:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761231.322890:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761231.322894:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1166 00000100:00080000:1.0:1789761231.322897:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761231.322901:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761231.322911:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761231.322922:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565652864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761231.322930:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565652992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761231.322944:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761231.322947:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761231.322950:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761231.322954:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1166 00000100:00080000:1.0:1789761231.322959:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761231.322963:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761231.322970:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789761231.323017:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761231.323049:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565653120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761231.323062:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761231.323067:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:3.0:1789761231.323070:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761231/real 1789761231] req@ffff93f844ccb480 x1876699565652864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761231.323071:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761231.323082:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1166 00000100:00080000:1.0:1789761231.323089:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1789761231.323096:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565652864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761231.323097:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761231.323109:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00080000:3.0:1789761231.323112:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761231.323119:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761231.323137:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565652864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761231.323140:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565653248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761231.323172:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565652992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761231.323202:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761231.323217:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761231/real 1789761231] req@ffff93f844ccaa00 x1876699565652992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761231.323232:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565652992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761231.323242:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761231.323245:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761231.323249:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565652992:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761231.323256:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565653120:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761231.323280:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761231.323290:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761231/real 1789761231] req@ffff93f844cc8000 x1876699565653120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761231.323300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565653120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761231.323306:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761231.323308:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761231.323311:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565653120:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761231.323315:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565653248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761231.323328:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761231.323334:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761231/real 1789761231] req@ffff93f844ccb800 x1876699565653248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761231.323342:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565653248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761231.323348:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761231.323350:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761231.323352:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565653248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761231.386794:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761232.410836:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761233.434790:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761234.458822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761235.482841:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761236.442752:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761236.442763:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761236.442768:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761236.442779:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1171 00000100:00080000:1.0:1789761236.442790:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761236.442798:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761236.442842:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565653376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761236.442866:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761236.442869:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761236.442871:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761236.442875:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1171 00000100:00080000:1.0:1789761236.442879:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761236.442885:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761236.442896:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761236.442911:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565653376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761236.442913:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565653504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761236.442929:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761236.442933:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761236.442935:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761236.442940:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1171 00000100:00080000:1.0:1789761236.442946:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761236.442952:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761236.442958:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761236.442973:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565653632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761236.442985:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789761236.442986:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761236.442992:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761236.442994:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761236.443001:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1171 00000100:00080000:1.0:1789761236.443007:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761236.443012:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761236.443019:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761236.443035:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565653760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761236.443038:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761236/real 1789761236] req@ffff93f844ccb100 x1876699565653376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761236.443071:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565653376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761236.443080:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761236.443096:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761236.443100:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565653376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761236.443124:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565653504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761236.443143:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761236.443154:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761236/real 1789761236] req@ffff93f844cc9c00 x1876699565653504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761236.443168:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565653504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761236.443177:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761236.443182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761236.443185:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565653504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761236.443192:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565653632:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761236.443223:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761236.443236:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761236/real 1789761236] req@ffff93f844cc8e00 x1876699565653632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761236.443250:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565653632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761236.443260:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761236.443264:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761236.443266:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565653632:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761236.443271:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565653760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761236.443288:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761236.443295:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761236/real 1789761236] req@ffff93f844cc9500 x1876699565653760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761236.443303:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565653760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761236.443312:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761236.443316:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761236.443319:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565653760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761236.506832:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761237.530796:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761238.554877:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761239.578763:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761240.602814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761241.562799:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761241.562807:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761241.562811:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761241.562821:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1176 00000100:00080000:1.0:1789761241.562830:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761241.562837:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761241.562872:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565653888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761241.562891:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761241.562895:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761241.562896:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761241.562900:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1176 00000100:00080000:1.0:1789761241.562903:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761241.562907:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761241.562916:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761241.562927:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565653888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761241.562931:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565654016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761241.562960:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761241.562962:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761241.562963:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761241.562966:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1176 00000100:00080000:1.0:1789761241.562970:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761241.562972:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761241.562976:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761241.562986:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565654144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761241.562994:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761241.562997:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761241.562999:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761241.563001:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1176 00000100:00080000:1.0:1789761241.563004:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761241.563007:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761241.563011:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761241.563020:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565654272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761241.563029:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761241.563107:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761241/real 1789761241] req@ffff93f844ccad80 x1876699565653888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761241.563123:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565653888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761241.563133:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761241.563137:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761241.563140:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565653888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761241.563187:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565654016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761241.563235:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761241.563252:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761241/real 1789761241] req@ffff93f844cca680 x1876699565654016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761241.563264:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565654016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761241.563272:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761241.563275:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761241.563278:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565654016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761241.563302:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565654144:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761241.563326:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761241.563336:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761241/real 1789761241] req@ffff93f844cc9500 x1876699565654144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761241.563346:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565654144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761241.563352:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761241.563354:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761241.563357:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565654144:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761241.563362:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565654272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761241.563427:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761241.563436:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761241/real 1789761241] req@ffff93f844cc8e00 x1876699565654272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761241.563444:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565654272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761241.563449:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761241.563451:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761241.563453:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565654272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761241.626800:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761242.650781:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761243.674850:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761244.698781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761245.722770:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761246.682776:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761246.682788:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761246.682792:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761246.682800:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1181 00000100:00080000:2.0:1789761246.682808:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761246.682815:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761246.682850:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565654400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761246.682872:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761246.682875:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761246.682876:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761246.682879:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1181 00000100:00080000:2.0:1789761246.682881:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761246.682883:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761246.682890:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761246.682901:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565654528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761246.682909:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761246.682911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761246.682912:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761246.682915:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1181 00000100:00080000:2.0:1789761246.682918:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761246.682921:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761246.682929:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761246.682939:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565654656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761246.682940:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565654400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761246.682966:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761246.682968:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761246.682969:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761246.682972:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1181 00000100:00080000:2.0:1789761246.682974:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761246.682976:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761246.682979:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761246.682990:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565654784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761246.683068:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761246.683119:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761246/real 1789761246] req@ffff93f8517c1880 x1876699565654400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761246.683136:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565654400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761246.683147:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761246.683151:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761246.683155:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565654400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761246.683161:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565654528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761246.683180:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761246.683189:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761246/real 1789761246] req@ffff93f8517c3480 x1876699565654528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761246.683197:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565654528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761246.683203:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761246.683206:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761246.683208:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565654528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761246.683233:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565654656:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761246.683245:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761246.683252:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761246/real 1789761246] req@ffff93f8517c3800 x1876699565654656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761246.683260:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565654656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761246.683265:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761246.683268:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761246.683270:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565654656:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761246.683275:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565654784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761246.683286:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761246.683292:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761246/real 1789761246] req@ffff93f8517c0e00 x1876699565654784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761246.683300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565654784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761246.683305:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761246.683307:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761246.683309:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565654784:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761246.746821:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761247.770834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761248.794826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761249.818848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761250.842812:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761251.802779:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761251.802787:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761251.802790:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761251.802799:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1186 00000100:00080000:2.0:1789761251.802808:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761251.802814:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761251.802850:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565654912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761251.802877:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761251.802884:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761251.802887:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761251.802893:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1186 00000100:00080000:2.0:1789761251.802906:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761251.802917:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761251.802926:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565654912:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789761251.802932:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761251.802952:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565655040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761251.802965:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761251.802985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761251.802987:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761251.802993:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1186 00000100:00080000:2.0:1789761251.803006:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761251.803011:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761251.803018:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761251.803033:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565655168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761251.803044:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761251.803049:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761251.803052:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761251.803059:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1186 00000100:00080000:2.0:1789761251.803063:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761251.803068:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761251.803075:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761251.803086:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565655040:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761251.803087:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565655296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761251.803146:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565655168:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761251.803171:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565655296:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761251.866839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761252.890763:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761253.914747:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789761254.875024:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789761254.875087:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789761254.875109:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761254.875169:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761254.875181:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761254.875190:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1789761254.875298:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761251/real 1789761254] req@ffff93f8517c1880 x1876699565655296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761254.875339:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565655296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761254.875355:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761254.875362:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761254.875369:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565655296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761254.875378:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761251/real 1789761254] req@ffff93f8517c3800 x1876699565655040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761254.875391:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565655040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761254.875401:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761254.875405:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761254.875408:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565655040:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761254.875416:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761251/real 1789761254] req@ffff93f8517c3480 x1876699565655168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761254.875430:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565655168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761254.875439:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761254.875442:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761254.875446:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565655168:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761254.875452:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761251/real 1789761254] req@ffff93f8517c0e00 x1876699565654912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761254.875464:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565654912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761254.875472:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761254.875477:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761254.875480:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565654912:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761254.938825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761255.962812:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761256.922761:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761256.922771:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761256.922776:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761256.922786:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1191 00000100:00080000:2.0:1789761256.922798:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761256.922807:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761256.922855:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565655424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761256.922884:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761256.922888:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761256.922890:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761256.922895:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1191 00000100:00080000:2.0:1789761256.922899:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761256.922903:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761256.922913:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761256.922927:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565655552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761256.922954:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761256.922956:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761256.922959:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1789761256.922961:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565655424:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761256.922964:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1191 00000100:00080000:2.0:1789761256.922971:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761256.922975:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761256.922980:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761256.922994:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565655680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761256.923004:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761256.923008:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761256.923010:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761256.923014:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1191 00000100:00080000:2.0:1789761256.923019:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761256.923023:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761256.923028:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789761256.923033:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761256.923045:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565655808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761256.923085:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761256/real 1789761256] req@ffff93f8517c2d80 x1876699565655424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761256.923099:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565655424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761256.923106:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761256.923109:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761256.923113:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565655424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761256.923139:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565655552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761256.923164:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761256.923178:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761256/real 1789761256] req@ffff93f8517c2a00 x1876699565655552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761256.923191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565655552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761256.923200:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761256.923203:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761256.923207:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565655552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761256.923216:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565655680:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761256.923240:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761256.923252:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761256/real 1789761256] req@ffff93f8517c3100 x1876699565655680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761256.923264:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565655680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761256.923272:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761256.923275:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761256.923278:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565655680:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761256.923284:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565655808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761256.923305:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761256.923316:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761256/real 1789761256] req@ffff93f8517c1180 x1876699565655808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761256.923329:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565655808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761256.923339:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761256.923343:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761256.923346:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565655808:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761256.986777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761258.010776:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761259.034809:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761260.058793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761261.082765:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761262.042760:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761262.042775:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761262.042781:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761262.042796:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1197 00000100:00080000:1.0:1789761262.042808:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761262.042818:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761262.042877:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565655936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761262.042921:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761262.042926:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761262.042929:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761262.042935:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1197 00000100:00080000:1.0:1789761262.042940:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761262.042946:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761262.042954:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565655936:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789761262.042958:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761262.042974:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565656064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761262.042982:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761262.042985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761262.042987:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761262.042990:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1197 00000100:00080000:1.0:1789761262.042993:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761262.042996:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761262.043001:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761262.043008:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565656192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761262.043016:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761262.043019:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761262.043021:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1789761262.043025:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789761262.043029:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1197 00000100:00080000:1.0:1789761262.043032:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761262.043035:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761262.043039:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761262.043048:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565656320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761262.043076:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761262/real 1789761262] req@ffff93f844cc9c00 x1876699565655936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761262.043098:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565655936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761262.043110:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761262.043115:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761262.043120:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565655936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761262.043146:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565656064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761262.043170:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761262.043184:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761262/real 1789761262] req@ffff93f844ccb100 x1876699565656064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761262.043198:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565656064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761262.043208:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761262.043212:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761262.043215:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565656064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761262.043223:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565656192:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761262.043243:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761262.043252:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761262/real 1789761262] req@ffff93f844ccb800 x1876699565656192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761262.043265:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565656192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761262.043271:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761262.043273:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761262.043275:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565656192:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761262.043281:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565656320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761262.043300:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761262.043311:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761262/real 1789761262] req@ffff93f844cc8000 x1876699565656320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761262.043324:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565656320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761262.043333:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761262.043337:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761262.043341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565656320:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761262.106850:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761263.130836:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761264.154807:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761265.178807:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761266.202831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761267.162789:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761267.162798:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761267.162802:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761267.162812:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1202 00000100:00080000:2.0:1789761267.162822:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761267.162827:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761267.162873:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565656448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761267.162896:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761267.162901:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761267.162903:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761267.162906:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1202 00000100:00080000:2.0:1789761267.162909:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761267.162911:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761267.162919:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761267.162930:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565656576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761267.162938:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761267.162940:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761267.162942:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761267.162945:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1202 00000100:00080000:2.0:1789761267.162947:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761267.162950:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761267.162954:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761267.162960:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565656704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761267.162971:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761267.162974:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761267.162975:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761267.162979:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1202 00000100:00080000:2.0:1789761267.162983:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1789761267.162985:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565656448:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761267.162986:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761267.162999:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761267.163015:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565656832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761267.163076:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761267.163127:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761267/real 1789761267] req@ffff93f8517c0a80 x1876699565656448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761267.163151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565656448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761267.163161:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761267.163165:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761267.163168:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565656448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761267.163175:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565656576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761267.163191:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761267.163200:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761267/real 1789761267] req@ffff93f8517c1500 x1876699565656576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761267.163208:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565656576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761267.163213:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761267.163216:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761267.163217:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565656576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761267.163241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565656704:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761267.163254:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761267.163261:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761267/real 1789761267] req@ffff93f8517c0380 x1876699565656704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761267.163268:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565656704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761267.163278:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761267.163280:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761267.163283:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565656704:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761267.163286:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565656832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761267.163297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761267.163303:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761267/real 1789761267] req@ffff93f8517c2680 x1876699565656832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761267.163311:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565656832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761267.163316:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761267.163317:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761267.163319:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565656832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761267.226799:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761268.250815:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761269.274756:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761270.298921:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761271.322927:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761272.282721:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761272.282730:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761272.282736:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761272.282744:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1207 00000100:00080000:2.0:1789761272.282751:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761272.282758:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761272.282794:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565656960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761272.282828:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761272.282831:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761272.282834:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761272.282837:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1207 00000100:00080000:2.0:1789761272.282841:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761272.282845:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761272.282853:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761272.282868:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565656960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761272.282868:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565657088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761272.282885:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761272.282889:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761272.282890:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761272.282895:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1207 00000100:00080000:2.0:1789761272.282900:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761272.282905:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761272.282911:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761272.282926:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565657216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761272.282939:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1789761272.282945:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761272.282945:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761272.282948:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761272.282955:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1207 00000100:00080000:2.0:1789761272.282960:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761272.282963:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761272.282968:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761272.282978:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565657344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761272.283002:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761272/real 1789761272] req@ffff93f8517c2300 x1876699565656960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761272.283028:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565656960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761272.283041:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761272.283046:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761272.283054:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565656960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761272.283080:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565657088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761272.283104:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761272.283118:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761272/real 1789761272] req@ffff93f8517c1f80 x1876699565657088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761272.283130:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565657088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761272.283148:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761272.283151:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761272.283155:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565657088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761272.283162:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565657216:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761272.283182:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761272.283193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761272/real 1789761272] req@ffff93f8517c3b80 x1876699565657216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761272.283211:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565657216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761272.283219:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761272.283222:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761272.283226:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565657216:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761272.283231:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565657344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761272.283251:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761272.283263:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761272/real 1789761272] req@ffff93f8517c0000 x1876699565657344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761272.283274:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565657344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761272.283282:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761272.283285:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761272.283288:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565657344:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761272.346883:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761273.370825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761274.394774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761275.418775:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761276.442801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761277.402733:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761277.402741:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761277.402745:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761277.402753:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1212 00000100:00080000:2.0:1789761277.402761:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761277.402767:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761277.402815:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565657472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761277.402837:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761277.402841:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761277.402843:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761277.402845:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1212 00000100:00080000:2.0:1789761277.402848:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761277.402851:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761277.402858:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761277.402867:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565657600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761277.402875:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789761277.402880:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565657472:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761277.402880:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761277.402884:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761277.402891:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1212 00000100:00080000:2.0:1789761277.402894:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761277.402896:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761277.402899:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761277.402908:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565657728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761277.402915:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761277.402918:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761277.402919:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761277.402922:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1212 00000100:00080000:2.0:1789761277.402924:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761277.402926:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761277.402929:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761277.402937:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565657856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761277.402944:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761277.402996:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761277/real 1789761277] req@ffff93f8517c0000 x1876699565657472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761277.403017:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565657472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761277.403037:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761277.403040:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761277.403044:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565657472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761277.403068:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565657600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761277.403093:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761277.403108:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761277/real 1789761277] req@ffff93f8517c3b80 x1876699565657600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761277.403119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565657600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761277.403136:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761277.403138:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761277.403142:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565657600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761277.403149:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565657728:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761277.403177:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761277.403195:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761277/real 1789761277] req@ffff93f8517c1f80 x1876699565657728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761277.403206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565657728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761277.403213:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761277.403216:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761277.403219:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565657728:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761277.403224:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565657856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761277.403242:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761277.403255:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761277/real 1789761277] req@ffff93f8517c2300 x1876699565657856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761277.403271:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565657856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761277.403277:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761277.403279:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761277.403282:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565657856:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761277.466795:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761278.490759:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761279.514845:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761280.538806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761281.562884:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761282.522795:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761282.522807:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761282.522811:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761282.522822:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1217 00000100:00080000:2.0:1789761282.522832:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761282.522839:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761282.522882:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565657984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761282.522908:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761282.522912:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761282.522913:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761282.522916:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1217 00000100:00080000:2.0:1789761282.522919:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761282.522922:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761282.522932:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761282.522941:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565658112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761282.522951:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789761282.522955:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565657984:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761282.522957:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761282.522962:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761282.522973:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1217 00000100:00080000:2.0:1789761282.522979:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761282.522985:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761282.522992:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761282.523009:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565658240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761282.523018:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761282.523021:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761282.523022:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761282.523026:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1217 00000800:00000100:0.0:1789761282.523032:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761282.523043:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761282.523051:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761282.523059:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761282.523078:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761282/real 1789761282] req@ffff93f8517c2680 x1876699565657984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761282.523082:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565658368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761282.523128:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565657984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761282.523144:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761282.523149:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761282.523152:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565657984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761282.523175:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565658112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761282.523194:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761282.523203:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761282/real 1789761282] req@ffff93f8517c0380 x1876699565658112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761282.523212:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565658112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761282.523218:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761282.523220:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761282.523222:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565658112:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761282.523226:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565658240:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761282.523237:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761282.523244:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761282/real 1789761282] req@ffff93f8517c1500 x1876699565658240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761282.523251:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565658240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761282.523256:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761282.523258:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761282.523260:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565658240:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761282.523263:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565658368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761282.523274:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761282.523279:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761282/real 1789761282] req@ffff93f8517c0a80 x1876699565658368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761282.523286:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565658368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761282.523296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761282.523298:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761282.523300:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565658368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761282.586819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761283.610803:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761284.634836:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761285.658830:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761286.682832:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761287.642800:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761287.642809:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761287.642813:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761287.642820:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1222 00000100:00080000:2.0:1789761287.642831:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761287.642837:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761287.642875:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565658496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761287.642902:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761287.642909:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761287.642911:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761287.642918:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1222 00000100:00080000:2.0:1789761287.642923:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761287.642929:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761287.642941:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761287.642943:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565658496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761287.642962:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565658624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761287.642977:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761287.642981:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761287.642983:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761287.642988:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1222 00000100:00080000:2.0:1789761287.642994:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761287.642998:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761287.643005:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789761287.643018:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761287.643023:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565658752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761287.643039:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761287.643043:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761287.643045:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761287.643049:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1222 00000100:00080000:2.0:1789761287.643053:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761287.643057:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761287.643063:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761287.643072:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565658880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761287.643087:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761287/real 1789761287] req@ffff93f8517c1180 x1876699565658496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761287.643112:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565658496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761287.643126:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761287.643131:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761287.643135:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565658496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761287.643166:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565658624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761287.643190:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761287.643201:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761287/real 1789761287] req@ffff93f8517c3100 x1876699565658624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761287.643209:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565658624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761287.643216:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761287.643219:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761287.643221:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565658624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761287.643227:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565658752:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761287.643240:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761287.643247:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761287/real 1789761287] req@ffff93f8517c2a00 x1876699565658752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761287.643255:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565658752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761287.643261:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761287.643263:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761287.643265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565658752:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761287.643269:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565658880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761287.643289:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761287.643301:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761287/real 1789761287] req@ffff93f8517c2d80 x1876699565658880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761287.643314:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565658880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761287.643323:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761287.643327:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761287.643329:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565658880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761287.706828:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761288.730854:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761289.754836:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761290.778860:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761291.802805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761292.762767:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761292.762776:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761292.762780:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761292.762788:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1227 00000100:00080000:1.0:1789761292.762797:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761292.762803:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761292.762836:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565659008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761292.762856:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761292.762859:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761292.762860:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761292.762864:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1227 00000100:00080000:1.0:1789761292.762866:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761292.762868:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761292.762877:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761292.762890:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565659136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761292.762900:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761292.762902:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761292.762904:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761292.762910:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1227 00000100:00080000:1.0:1789761292.762916:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789761292.762917:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565659008:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761292.762921:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761292.762933:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761292.762940:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565659264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761292.762948:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761292.762950:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761292.762952:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761292.762956:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1227 00000100:00080000:1.0:1789761292.762958:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761292.762960:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761292.762963:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761292.762971:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565659392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761292.763031:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761292.763081:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761292/real 1789761292] req@ffff93f844ccaa00 x1876699565659008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761292.763105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565659008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761292.763118:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761292.763124:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761292.763128:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565659008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761292.763160:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565659136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761292.763187:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761292.763203:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761292/real 1789761292] req@ffff93f844ccb480 x1876699565659136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761292.763216:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565659136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761292.763227:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761292.763229:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761292.763232:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565659136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761292.763237:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565659264:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761292.763256:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761292.763265:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761292/real 1789761292] req@ffff93f844cc8700 x1876699565659264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761292.763273:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565659264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761292.763283:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761292.763285:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761292.763288:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565659264:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761292.763291:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565659392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761292.763303:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761292.763309:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761292/real 1789761292] req@ffff93f844ccbb80 x1876699565659392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761292.763317:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565659392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761292.763322:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761292.763325:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761292.763328:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565659392:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761292.826828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761293.850802:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761294.874810:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761295.898778:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761296.922814:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761297.883363:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761297.883373:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761297.883377:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761297.883389:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1232 00000100:00080000:2.0:1789761297.883399:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761297.883410:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761297.883448:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565659520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761297.883474:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761297.883477:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761297.883479:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761297.883482:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1232 00000100:00080000:2.0:1789761297.883487:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761297.883492:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761297.883503:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761297.883514:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565659648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761297.883527:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565659520:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761297.883529:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761297.883545:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761297.883548:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761297.883552:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1232 00000100:00080000:2.0:1789761297.883555:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761297.883559:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761297.883564:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761297.883572:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565659776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761297.883580:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761297.883583:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761297.883584:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761297.883588:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1232 00000100:00080000:2.0:1789761297.883593:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761297.883598:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761297.883603:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761297.883622:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565659904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761297.883636:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761297.883756:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761297/real 1789761297] req@ffff93f8517c0e00 x1876699565659520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761297.883775:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565659520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761297.883785:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761297.883789:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761297.883795:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565659520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761297.883822:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565659648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761297.883843:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761297.883853:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761297/real 1789761297] req@ffff93f8517c3480 x1876699565659648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761297.883862:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565659648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761297.883868:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761297.883871:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761297.883873:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565659648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761297.883878:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565659776:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761297.883890:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761297.883897:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761297/real 1789761297] req@ffff93f8517c3800 x1876699565659776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761297.883905:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565659776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761297.883911:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761297.883913:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761297.883914:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565659776:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761297.883918:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565659904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761297.883930:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761297.883936:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761297/real 1789761297] req@ffff93f8517c1880 x1876699565659904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761297.883945:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565659904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761297.883951:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761297.883952:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761297.883954:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565659904:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761297.946787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761298.970818:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761299.994785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761301.018824:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761302.042819:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761303.002773:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761303.002781:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761303.002785:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761303.002796:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1237 00000100:00080000:2.0:1789761303.002805:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761303.002812:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761303.002845:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565660032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761303.002870:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761303.002875:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761303.002876:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761303.002879:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1237 00000100:00080000:2.0:1789761303.002882:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761303.002884:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761303.002892:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761303.002903:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565660160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761303.002910:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761303.002912:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761303.002914:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761303.002916:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1237 00000100:00080000:2.0:1789761303.002918:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761303.002921:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761303.002923:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761303.002934:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565660032:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761303.002934:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565660288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761303.002957:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761303.002960:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761303.002961:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761303.002963:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1237 00000100:00080000:2.0:1789761303.002965:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761303.002967:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761303.002971:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761303.002979:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565660416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761303.003039:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761303.003104:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761303/real 1789761303] req@ffff93f8517c1880 x1876699565660032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761303.003122:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565660032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761303.003132:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761303.003136:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761303.003141:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565660032:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761303.003151:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565660160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761303.003186:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761303.003200:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761303/real 1789761303] req@ffff93f8517c3800 x1876699565660160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761303.003209:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565660160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761303.003220:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761303.003223:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761303.003225:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565660160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761303.003252:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565660288:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761303.003276:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761303.003284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761303/real 1789761303] req@ffff93f8517c3480 x1876699565660288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761303.003292:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565660288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761303.003298:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761303.003301:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761303.003303:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565660288:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761303.003312:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565660416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761303.003326:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761303.003334:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761303/real 1789761303] req@ffff93f8517c0e00 x1876699565660416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761303.003342:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565660416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761303.003347:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761303.003350:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761303.003352:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565660416:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761303.066788:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761304.090818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761305.114780:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761306.138767:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761307.162814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761308.122759:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761308.122770:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761308.122775:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761308.122787:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1243 00000100:00080000:1.0:1789761308.122797:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761308.122805:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761308.122848:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565660544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761308.122876:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761308.122878:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761308.122880:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761308.122883:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1243 00000100:00080000:1.0:1789761308.122886:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761308.122889:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761308.122898:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761308.122907:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565660544:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761308.122913:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565660672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761308.122922:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761308.122924:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761308.122925:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761308.122928:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1243 00000100:00080000:1.0:1789761308.122930:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761308.122934:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761308.122937:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761308.122947:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565660800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761308.122955:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761308.122958:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761308.122959:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761308.122961:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1243 00000100:00080000:1.0:1789761308.122966:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761308.122969:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761308.122971:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789761308.122978:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761308.122981:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565660928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761308.123031:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761308/real 1789761308] req@ffff93f844cc8a80 x1876699565660544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761308.123047:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565660544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761308.123058:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761308.123061:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761308.123065:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565660544:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761308.123086:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565660672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761308.123102:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761308.123111:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761308/real 1789761308] req@ffff93f844cc8380 x1876699565660672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761308.123119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565660672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761308.123124:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761308.123126:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761308.123128:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565660672:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761308.123132:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565660800:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761308.123150:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761308.123158:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761308/real 1789761308] req@ffff93f844cca300 x1876699565660800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761308.123166:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565660800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761308.123171:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761308.123173:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761308.123175:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565660800:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761308.123178:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565660928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761308.123189:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761308.123196:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761308/real 1789761308] req@ffff93f844cc9180 x1876699565660928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761308.123203:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565660928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761308.123208:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761308.123210:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761308.123212:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565660928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761308.186796:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761309.210811:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761310.234844:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761311.258847:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761312.282884:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761313.242760:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761313.242774:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761313.242779:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761313.242794:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1248 00000100:00080000:0.0:1789761313.242810:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761313.242821:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761313.242870:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565661056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761313.242904:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761313.242909:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761313.242911:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761313.242917:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1248 00000100:00080000:0.0:1789761313.242925:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761313.242930:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761313.242941:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761313.242952:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565661056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761313.242967:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565661184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761313.242979:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761313.242983:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761313.242985:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761313.242992:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1248 00000100:00080000:0.0:1789761313.242999:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761313.243003:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761313.243010:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761313.243026:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565661312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761313.243039:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:1.0:1789761313.243041:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789761313.243047:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761313.243051:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761313.243060:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1248 00000100:00080000:0.0:1789761313.243072:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761313.243077:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761313.243086:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761313.243098:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761313/real 1789761313] req@ffff93f850d40e00 x1876699565661056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761313.243107:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565661440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761313.243120:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565661056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761313.243136:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761313.243143:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761313.243149:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565661056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761313.243179:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565661184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761313.243206:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761313.243222:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761313/real 1789761313] req@ffff93f850d41c00 x1876699565661184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761313.243237:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565661184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761313.243246:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761313.243250:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761313.243255:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565661184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761313.243262:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565661312:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761313.243289:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761313.243303:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761313/real 1789761313] req@ffff93f850d42680 x1876699565661312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761313.243317:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565661312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761313.243327:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761313.243330:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761313.243334:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565661312:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761313.243342:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565661440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761313.243368:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761313.243382:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761313/real 1789761313] req@ffff93f850d40000 x1876699565661440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761313.243395:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565661440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761313.243404:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761313.243408:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761313.243413:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565661440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761313.306805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761314.330789:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761315.354792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761316.378821:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761317.402772:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761318.362754:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761318.362762:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761318.362765:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761318.362773:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1253 00000100:00080000:1.0:1789761318.362781:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761318.362787:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761318.362820:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565661568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761318.362844:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761318.362847:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761318.362848:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761318.362851:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1253 00000100:00080000:1.0:1789761318.362853:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761318.362856:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761318.362864:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761318.362874:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565661696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761318.362882:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761318.362885:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1789761318.362886:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565661568:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761318.362887:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761318.362896:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1253 00000100:00080000:1.0:1789761318.362898:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761318.362901:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761318.362904:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761318.362914:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565661824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761318.362921:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761318.362924:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761318.362925:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761318.362927:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1253 00000100:00080000:1.0:1789761318.362944:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761318.362946:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761318.362949:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761318.362956:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565661952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761318.363014:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565661696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761318.363056:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565661824:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761318.363075:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565661952:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761318.426821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761319.450826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761320.474804:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789761321.435008:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789761321.435067:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789761321.435088:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761321.435149:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761321.435163:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761321.435173:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789761321.435269:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761318/real 1789761321] req@ffff93f844cc9f80 x1876699565661696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761321.435306:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565661696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761321.435321:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761321.435327:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761321.435331:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565661696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761321.435341:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761318/real 1789761321] req@ffff93f844cc9180 x1876699565661824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761321.435353:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565661824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761321.435363:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761321.435367:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761321.435370:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565661824:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761321.435377:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761318/real 1789761321] req@ffff93f844cca300 x1876699565661952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761321.435389:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565661952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761321.435397:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761321.435401:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761321.435404:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565661952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761321.435411:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761318/real 1789761321] req@ffff93f844cc9880 x1876699565661568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761321.435423:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565661568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761321.435430:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761321.435434:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761321.435437:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565661568:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1789761321.498825:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761322.522832:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761323.482795:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761323.482809:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761323.482815:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761323.482829:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1258 00000100:00080000:0.0:1789761323.482843:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761323.482853:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761323.482901:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565662080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761323.482939:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761323.482944:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761323.482947:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761323.482952:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1258 00000100:00080000:0.0:1789761323.482957:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761323.482963:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761323.482974:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761323.482979:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565662080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761323.482995:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565662208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761323.483010:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761323.483024:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761323.483027:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761323.483032:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1258 00000100:00080000:0.0:1789761323.483047:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761323.483053:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761323.483060:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:1.0:1789761323.483062:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761323.483079:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565662336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761323.483093:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761323.483098:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761323.483101:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761323.483105:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1258 00000100:00080000:0.0:1789761323.483110:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761323.483116:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761323.483122:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761323/real 1789761323] req@ffff93f850d41180 x1876699565662080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1789761323.483123:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761323.483142:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565662464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761323.483151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565662080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761323.483168:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761323.483174:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761323.483179:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565662080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761323.483211:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565662208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761323.483239:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761323.483254:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761323/real 1789761323] req@ffff93f850d43100 x1876699565662208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761323.483266:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565662208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761323.483274:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761323.483278:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761323.483281:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565662208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761323.483288:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565662336:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761323.483309:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761323.483321:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761323/real 1789761323] req@ffff93f850d41f80 x1876699565662336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761323.483347:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565662336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761323.483357:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761323.483360:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761323.483363:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565662336:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761323.483371:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565662464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761323.483393:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761323.483406:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761323/real 1789761323] req@ffff93f850d41500 x1876699565662464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761323.483417:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565662464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761323.483425:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761323.483428:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761323.483446:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565662464:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761323.546776:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761324.570843:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761325.594779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761326.618811:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761327.642792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761328.602760:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761328.602768:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761328.602772:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761328.602780:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1263 00000100:00080000:1.0:1789761328.602788:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761328.602794:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761328.602824:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565662592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761328.602847:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761328.602850:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761328.602851:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761328.602854:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1263 00000100:00080000:1.0:1789761328.602856:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761328.602859:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761328.602867:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761328.602875:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565662720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761328.602884:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761328.602886:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761328.602887:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761328.602893:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1263 00000100:00080000:1.0:1789761328.602898:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789761328.602899:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565662592:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761328.602902:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761328.602914:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761328.602923:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565662848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761328.602932:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761328.602934:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761328.602935:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761328.602939:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1263 00000100:00080000:1.0:1789761328.602941:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761328.602943:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761328.602946:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761328.602953:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565662976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761328.602992:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761328.603038:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761328/real 1789761328] req@ffff93f844cc8380 x1876699565662592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761328.603053:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565662592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761328.603067:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761328.603071:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761328.603074:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565662592:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761328.603100:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565662720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761328.603115:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761328.603124:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761328/real 1789761328] req@ffff93f844cc8a80 x1876699565662720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761328.603133:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565662720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761328.603139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761328.603142:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761328.603144:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565662720:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761328.603148:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565662848:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761328.603160:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761328.603166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761328/real 1789761328] req@ffff93f844ccbb80 x1876699565662848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761328.603174:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565662848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761328.603179:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761328.603181:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761328.603183:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565662848:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761328.603187:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565662976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761328.603198:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761328.603205:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761328/real 1789761328] req@ffff93f844cc8700 x1876699565662976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761328.603212:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565662976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761328.603217:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761328.603219:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761328.603222:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565662976:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761328.666804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761329.690780:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761330.714821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761331.738817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761332.762789:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761333.722749:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761333.722760:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761333.722765:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761333.722775:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1268 00000100:00080000:1.0:1789761333.722786:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761333.722793:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761333.722845:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565663104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761333.722871:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761333.722875:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761333.722876:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761333.722879:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1268 00000100:00080000:1.0:1789761333.722882:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761333.722884:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761333.722892:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761333.722902:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565663232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761333.722914:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761333.722917:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761333.722918:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761333.722923:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1268 00000100:00080000:1.0:1789761333.722925:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789761333.722927:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565663104:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761333.722928:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761333.722935:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761333.722948:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565663360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761333.722956:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761333.722959:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761333.722960:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761333.722963:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1268 00000100:00080000:1.0:1789761333.722965:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761333.722968:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761333.722971:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761333.722983:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565663488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761333.723014:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761333.723074:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761333/real 1789761333] req@ffff93f844ccb480 x1876699565663104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761333.723098:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565663104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761333.723111:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761333.723116:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761333.723120:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565663104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761333.723149:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565663232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761333.723171:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761333.723181:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761333/real 1789761333] req@ffff93f844ccaa00 x1876699565663232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761333.723189:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565663232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761333.723196:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761333.723198:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761333.723200:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565663232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761333.723204:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565663360:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761333.723216:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761333.723231:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761333/real 1789761333] req@ffff93f844cc8000 x1876699565663360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761333.723239:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565663360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761333.723245:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761333.723247:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761333.723249:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565663360:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761333.723253:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565663488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761333.723264:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761333.723271:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761333/real 1789761333] req@ffff93f844ccb800 x1876699565663488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761333.723279:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565663488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761333.723284:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761333.723286:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761333.723288:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565663488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761333.786836:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761334.810838:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761335.834860:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761336.858778:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761337.882882:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761338.842795:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761338.842806:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761338.842810:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761338.842821:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1273 00000100:00080000:0.0:1789761338.842832:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761338.842840:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761338.842879:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565663616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761338.842907:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761338.842910:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761338.842912:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761338.842920:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1273 00000100:00080000:0.0:1789761338.842924:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761338.842929:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761338.842939:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761338.842958:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565663744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761338.842961:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565663616:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761338.842976:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761338.842980:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761338.842981:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761338.842985:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1273 00000100:00080000:0.0:1789761338.842988:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761338.842990:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761338.842993:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761338.843003:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565663872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761338.843011:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761338.843013:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761338.843015:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761338.843019:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1273 00000100:00080000:0.0:1789761338.843038:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761338.843041:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761338.843047:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789761338.843053:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761338.843060:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565664000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761338.843108:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761338/real 1789761338] req@ffff93f850d43480 x1876699565663616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761338.843139:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565663616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761338.843154:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761338.843161:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761338.843170:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565663616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761338.843198:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565663744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761338.843218:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761338.843228:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761338/real 1789761338] req@ffff93f850d43800 x1876699565663744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761338.843238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565663744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761338.843244:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761338.843247:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761338.843249:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565663744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761338.843253:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565663872:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761338.843266:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761338.843272:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761338/real 1789761338] req@ffff93f850d41500 x1876699565663872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761338.843280:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565663872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761338.843290:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761338.843292:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761338.843294:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565663872:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761338.843298:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565664000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761338.843312:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761338.843338:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761338/real 1789761338] req@ffff93f850d41f80 x1876699565664000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761338.843351:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565664000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761338.843361:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761338.843366:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761338.843369:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565664000:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761338.906829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761339.930762:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761340.954809:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761341.978858:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761343.002785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761343.962750:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761343.962764:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761343.962770:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761343.962782:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1278 00000100:00080000:1.0:1789761343.962793:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761343.962801:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761343.962842:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565664128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761343.962875:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761343.962878:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761343.962880:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761343.962883:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1278 00000100:00080000:1.0:1789761343.962886:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761343.962889:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761343.962898:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761343.962908:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565664256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761343.962913:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565664128:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761343.962921:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761343.962924:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761343.962925:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761343.962928:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1278 00000100:00080000:1.0:1789761343.962930:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761343.962932:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761343.962936:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761343.962944:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565664384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761343.962951:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761343.962953:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761343.962955:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761343.962958:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1278 00000100:00080000:1.0:1789761343.962983:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761343.962986:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761343.962989:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789761343.962997:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761343.963000:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565664512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761343.963060:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761343/real 1789761343] req@ffff93f844ccb100 x1876699565664128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761343.963080:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565664128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761343.963090:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761343.963093:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761343.963097:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565664128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761343.963121:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565664256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761343.963140:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761343.963156:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761343/real 1789761343] req@ffff93f844cc9c00 x1876699565664256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761343.963165:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565664256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761343.963171:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761343.963174:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761343.963177:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565664256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761343.963181:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565664384:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761343.963195:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761343.963202:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761343/real 1789761343] req@ffff93f844cc8e00 x1876699565664384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761343.963210:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565664384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761343.963216:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761343.963217:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761343.963219:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565664384:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761343.963223:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565664512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761343.963236:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761343.963242:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761343/real 1789761343] req@ffff93f844cc9500 x1876699565664512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761343.963250:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565664512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761343.963255:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761343.963256:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761343.963258:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565664512:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761344.026822:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761345.050805:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761346.074841:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761347.098821:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761348.122818:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761349.082777:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761349.082790:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761349.082795:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761349.082808:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1284 00000100:00080000:1.0:1789761349.082822:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761349.082831:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761349.082874:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565664640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761349.082913:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761349.082918:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761349.082920:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761349.082925:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1284 00000100:00080000:1.0:1789761349.082931:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761349.082936:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761349.082949:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761349.082966:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565664640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761349.082977:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565664768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761349.082998:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761349.083003:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761349.083006:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761349.083013:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1284 00000100:00080000:1.0:1789761349.083020:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761349.083026:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761349.083034:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789761349.083051:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761349.083058:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565664896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761349.083075:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761349.083081:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761349.083083:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761349.083091:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1284 00000100:00080000:1.0:1789761349.083098:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789761349.083103:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761349/real 1789761349] req@ffff93f844cca680 x1876699565664640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761349.083105:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761349.083120:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761349.083131:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565664640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761349.083143:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565665024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761349.083145:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761349.083153:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761349.083157:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565664640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761349.083194:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565664768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761349.083219:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761349.083229:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761349/real 1789761349] req@ffff93f844ccad80 x1876699565664768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761349.083257:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565664768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761349.083264:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761349.083267:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761349.083271:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565664768:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761349.083277:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565664896:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761349.083299:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761349.083313:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761349/real 1789761349] req@ffff93f844cc9500 x1876699565664896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761349.083323:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565664896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761349.083329:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761349.083332:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761349.083337:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565664896:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761349.083342:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565665024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761349.083364:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761349.083371:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761349/real 1789761349] req@ffff93f844cc8e00 x1876699565665024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761349.083384:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565665024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761349.083390:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761349.083394:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761349.083398:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565665024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761349.146807:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761350.170831:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761351.194838:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761352.218805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761353.242860:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761354.202719:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761354.202730:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761354.202735:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761354.202745:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1289 00000100:00080000:1.0:1789761354.202755:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761354.202763:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761354.202803:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565665152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761354.202828:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761354.202831:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761354.202832:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761354.202835:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1289 00000100:00080000:1.0:1789761354.202838:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761354.202841:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761354.202850:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761354.202863:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565665152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761354.202865:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565665280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761354.202878:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761354.202881:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761354.202883:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761354.202887:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1289 00000100:00080000:1.0:1789761354.202892:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761354.202896:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761354.202902:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761354.202916:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565665408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761354.202928:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761354.202933:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761354.202936:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789761354.202942:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789761354.202943:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1289 00000100:00080000:1.0:1789761354.202950:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761354.202955:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761354.202969:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761354.202984:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565665536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761354.202987:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761354/real 1789761354] req@ffff93f844cc9c00 x1876699565665152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761354.203030:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565665152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761354.203044:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761354.203050:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761354.203056:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565665152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761354.203085:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565665280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761354.203107:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761354.203118:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761354/real 1789761354] req@ffff93f844ccb100 x1876699565665280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761354.203127:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565665280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761354.203133:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761354.203135:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761354.203138:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565665280:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761354.203143:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565665408:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761354.203163:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761354.203176:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761354/real 1789761354] req@ffff93f844ccb800 x1876699565665408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761354.203190:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565665408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761354.203217:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761354.203220:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761354.203223:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565665408:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761354.203227:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565665536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761354.203244:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761354.203252:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761354/real 1789761354] req@ffff93f844cc8000 x1876699565665536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761354.203261:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565665536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761354.203266:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761354.203268:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761354.203270:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565665536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761354.266774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761355.290807:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761356.314758:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761357.338844:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761358.362823:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761359.322768:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761359.322779:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761359.322784:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761359.322798:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1294 00000100:00080000:1.0:1789761359.322812:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761359.322822:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761359.322892:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565665664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761359.322925:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761359.322930:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761359.322932:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761359.322937:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1294 00000100:00080000:1.0:1789761359.322941:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761359.322946:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761359.322972:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761359.322975:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565665664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761359.322991:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565665792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761359.323001:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761359.323003:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761359.323005:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761359.323009:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1294 00000100:00080000:1.0:1789761359.323012:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761359.323014:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761359.323018:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761359.323028:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565665920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761359.323036:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761359.323039:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761359.323040:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761359.323059:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1294 00000100:00080000:1.0:1789761359.323061:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761359.323064:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761359.323067:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761359.323075:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761359.323077:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565666048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761359.323135:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761359/real 1789761359] req@ffff93f844ccaa00 x1876699565665664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761359.323154:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565665664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761359.323178:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761359.323182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761359.323185:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565665664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761359.323210:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565665792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761359.323230:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761359.323240:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761359/real 1789761359] req@ffff93f844ccb480 x1876699565665792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761359.323248:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565665792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761359.323255:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761359.323257:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761359.323259:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565665792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761359.323263:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565665920:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761359.323275:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761359.323281:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761359/real 1789761359] req@ffff93f844cc8700 x1876699565665920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761359.323288:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565665920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761359.323299:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761359.323300:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761359.323303:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565665920:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761359.323307:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565666048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761359.323319:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761359.323326:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761359/real 1789761359] req@ffff93f844ccbb80 x1876699565666048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761359.323350:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565666048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761359.323356:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761359.323358:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761359.323360:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565666048:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761359.386857:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761360.410761:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761361.434821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761362.458787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761363.482786:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761364.443087:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761364.443096:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761364.443100:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761364.443109:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1299 00000100:00080000:3.0:1789761364.443118:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761364.443124:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761364.443173:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565666176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761364.443199:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761364.443204:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761364.443206:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761364.443210:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1299 00000100:00080000:3.0:1789761364.443215:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761364.443220:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761364.443230:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761364.443242:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565666176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761364.443248:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565666304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761364.443262:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761364.443267:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761364.443269:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761364.443274:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1299 00000100:00080000:3.0:1789761364.443279:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761364.443282:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761364.443287:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761364.443301:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565666432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761364.443313:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761364.443313:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761364.443320:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761364.443324:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761364.443330:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1299 00000100:00080000:3.0:1789761364.443335:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761364.443340:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761364.443345:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761364.443357:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761364/real 1789761364] req@ffff93f844d67800 x1876699565666176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761364.443362:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565666560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761364.443385:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565666176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761364.443395:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761364.443398:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761364.443408:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565666176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761364.443430:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565666304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761364.443449:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761364.443459:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761364/real 1789761364] req@ffff93f844d67480 x1876699565666304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761364.443473:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565666304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761364.443500:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761364.443504:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761364.443508:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565666304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761364.443515:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565666432:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761364.443545:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761364.443558:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761364/real 1789761364] req@ffff93f844d65880 x1876699565666432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761364.443571:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565666432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761364.443581:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761364.443585:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761364.443589:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565666432:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761364.443595:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565666560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761364.443618:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761364.443631:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761364/real 1789761364] req@ffff93f844d66680 x1876699565666560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761364.443671:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565666560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761364.443678:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761364.443680:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761364.443684:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565666560:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761364.506781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761365.530806:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761366.554774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761367.578815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761368.602816:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761369.562767:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761369.562778:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761369.562783:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761369.562794:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1304 00000100:00080000:2.0:1789761369.562804:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761369.562812:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761369.562855:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565666688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761369.562881:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761369.562884:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761369.562885:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761369.562888:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1304 00000100:00080000:2.0:1789761369.562890:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761369.562893:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761369.562901:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761369.562912:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565666816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761369.562920:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761369.562922:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761369.562924:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761369.562927:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1304 00000100:00080000:2.0:1789761369.562929:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761369.562932:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761369.562936:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761369.562941:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565666688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761369.562945:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565666944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761369.562958:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761369.562961:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761369.562962:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761369.562964:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1304 00000100:00080000:2.0:1789761369.562966:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761369.562968:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761369.562972:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761369.562979:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565667072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761369.563028:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761369.563081:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761369/real 1789761369] req@ffff93f8517c2d80 x1876699565666688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761369.563112:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565666688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761369.563122:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761369.563125:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761369.563129:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565666688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761369.563134:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565666816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761369.563153:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761369.563162:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761369/real 1789761369] req@ffff93f8517c2a00 x1876699565666816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761369.563173:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565666816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761369.563182:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761369.563187:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761369.563190:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565666816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761369.563222:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565666944:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761369.563246:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761369.563256:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761369/real 1789761369] req@ffff93f8517c3100 x1876699565666944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761369.563269:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565666944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761369.563275:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761369.563277:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761369.563279:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565666944:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761369.563284:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565667072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761369.563297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761369.563303:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761369/real 1789761369] req@ffff93f8517c1180 x1876699565667072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761369.563311:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565667072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761369.563316:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761369.563317:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761369.563319:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565667072:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761369.626798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761370.650793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761371.674806:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761372.698840:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761373.722784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761374.682748:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761374.682759:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761374.682764:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761374.682775:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1309 00000100:00080000:2.0:1789761374.682786:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761374.682794:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761374.682837:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565667200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761374.682868:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761374.682873:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761374.682876:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761374.682880:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1309 00000100:00080000:2.0:1789761374.682885:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761374.682890:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761374.682903:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761374.682919:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565667200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761374.682921:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565667328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761374.682936:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761374.682938:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761374.682940:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761374.682943:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1309 00000100:00080000:2.0:1789761374.682946:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761374.682949:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761374.682954:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761374.682965:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565667456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761374.682974:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761374.682977:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761374.682979:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761374.682981:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1309 00000100:00080000:2.0:1789761374.682984:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761374.682986:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761374.682989:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761374.682998:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565667584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761374.683010:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761374.683075:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761374/real 1789761374] req@ffff93f8517c0a80 x1876699565667200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761374.683095:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565667200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761374.683115:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761374.683120:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761374.683125:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565667200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761374.683153:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565667328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761374.683175:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761374.683186:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761374/real 1789761374] req@ffff93f8517c1500 x1876699565667328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761374.683194:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565667328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761374.683200:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761374.683203:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761374.683205:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565667328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761374.683209:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565667456:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761374.683221:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761374.683228:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761374/real 1789761374] req@ffff93f8517c0380 x1876699565667456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761374.683235:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565667456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761374.683241:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761374.683243:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761374.683245:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565667456:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761374.683249:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565667584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761374.683260:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761374.683266:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761374/real 1789761374] req@ffff93f8517c2680 x1876699565667584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761374.683274:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565667584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761374.683279:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761374.683281:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761374.683283:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565667584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761374.746829:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761375.770774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761376.794849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761377.818777:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761378.842839:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761379.802760:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761379.802769:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761379.802772:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761379.802781:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1314 00000100:00080000:1.0:1789761379.802791:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761379.802798:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761379.802832:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565667712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761379.802857:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761379.802862:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761379.802864:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761379.802868:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1314 00000100:00080000:1.0:1789761379.802872:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761379.802877:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761379.802888:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761379.802904:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565667840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761379.802917:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565667712:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761379.802922:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761379.802930:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761379.802932:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761379.802938:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1314 00000100:00080000:1.0:1789761379.802943:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761379.802947:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761379.802953:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761379.802963:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565667968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761379.802972:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761379.802974:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761379.802975:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761379.802978:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1314 00000100:00080000:1.0:1789761379.802980:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761379.802982:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761379.802985:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761379.802991:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565668096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761379.803047:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761379.803120:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761379/real 1789761379] req@ffff93f844cc8a80 x1876699565667712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761379.803146:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565667712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761379.803160:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761379.803165:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761379.803170:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565667712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761379.803203:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565667840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761379.803230:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761379.803244:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761379/real 1789761379] req@ffff93f844cc8380 x1876699565667840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761379.803257:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565667840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761379.803263:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761379.803265:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761379.803268:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565667840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761379.803272:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565667968:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761379.803289:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761379.803297:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761379/real 1789761379] req@ffff93f844cc9880 x1876699565667968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761379.803306:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565667968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761379.803311:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761379.803314:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761379.803316:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565667968:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761379.803320:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565668096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761379.803331:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761379.803337:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761379/real 1789761379] req@ffff93f844cca300 x1876699565668096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761379.803345:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565668096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761379.803355:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761379.803357:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761379.803359:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565668096:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789761379.866777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761380.890827:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761381.914761:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761382.938829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761383.962751:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761384.922744:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761384.922759:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761384.922764:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761384.922780:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1319 00000100:00080000:1.0:1789761384.922793:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761384.922801:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761384.922843:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565668224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761384.922872:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761384.922876:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761384.922879:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761384.922885:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1319 00000100:00080000:1.0:1789761384.922889:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761384.922893:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761384.922903:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761384.922906:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565668224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761384.922918:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565668352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761384.922929:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761384.922931:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761384.922932:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761384.922936:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1319 00000100:00080000:1.0:1789761384.922939:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761384.922942:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761384.922946:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761384.922959:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565668480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761384.922967:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761384.922971:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761384.922990:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761384.922998:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1319 00000100:00080000:1.0:1789761384.923004:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761384.923009:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761384.923017:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761384.923038:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565668608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761384.923110:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565668352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761384.923152:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565668480:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761384.923174:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565668608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761384.986813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789761386.010771:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761387.034792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789761387.994921:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789761387.994980:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789761387.995004:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761387.995042:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761387.995050:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761387.995058:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1789761387.995085:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761384/real 1789761387] req@ffff93f844cc9f80 x1876699565668352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761387.995112:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565668352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761387.995125:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761387.995130:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761387.995135:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565668352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761387.995143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761384/real 1789761387] req@ffff93f844cca300 x1876699565668480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761387.995174:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565668480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761387.995183:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761387.995187:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761387.995191:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565668480:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761387.995198:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761384/real 1789761387] req@ffff93f844cc9880 x1876699565668608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761387.995211:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565668608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761387.995220:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761387.995224:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761387.995232:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565668608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761387.995240:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761384/real 1789761387] req@ffff93f844cc9180 x1876699565668224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761387.995257:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565668224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761387.995266:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761387.995270:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761387.995274:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565668224:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761388.058767:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761389.082835:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761390.042762:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761390.042775:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761390.042780:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761390.042793:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1325 00000100:00080000:1.0:1789761390.042805:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761390.042815:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761390.042860:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565668736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761390.042898:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761390.042902:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761390.042905:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761390.042910:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1325 00000100:00080000:1.0:1789761390.042915:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761390.042920:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761390.042932:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761390.042951:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565668736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761390.042965:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565668864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761390.042979:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761390.042983:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761390.042985:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761390.042989:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1325 00000100:00080000:1.0:1789761390.042995:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761390.043000:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761390.043006:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761390.043020:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565668992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761390.043032:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761390.043039:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:3.0:1789761390.043040:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789761390.043042:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761390.043049:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1325 00000100:00080000:1.0:1789761390.043054:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761390.043058:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761390.043064:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761390.043076:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565669120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761390.043098:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761390/real 1789761390] req@ffff93f844cc8380 x1876699565668736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761390.043119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565668736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761390.043131:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761390.043135:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761390.043140:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565668736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761390.043169:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565668864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761390.043196:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761390.043225:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761390/real 1789761390] req@ffff93f844cc8a80 x1876699565668864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761390.043239:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565668864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761390.043249:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761390.043252:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761390.043256:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565668864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761390.043264:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565668992:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761390.043289:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761390.043303:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761390/real 1789761390] req@ffff93f844ccbb80 x1876699565668992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761390.043315:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565668992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761390.043323:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761390.043327:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761390.043330:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565668992:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761390.043336:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565669120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761390.043357:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761390.043368:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761390/real 1789761390] req@ffff93f844cc8700 x1876699565669120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761390.043379:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565669120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761390.043387:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761390.043390:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761390.043393:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565669120:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761390.106821:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761391.130781:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761392.154858:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761393.178766:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761394.202828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761395.162757:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761395.162768:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761395.162774:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761395.162785:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1330 00000100:00080000:1.0:1789761395.162795:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761395.162803:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761395.162845:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565669248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761395.162871:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761395.162877:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761395.162878:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761395.162881:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1330 00000100:00080000:1.0:1789761395.162884:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761395.162886:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761395.162908:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761395.162925:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565669376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761395.162935:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1789761395.162936:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565669248:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761395.162940:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761395.162945:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761395.162949:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1330 00000100:00080000:1.0:1789761395.162952:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761395.162955:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761395.162959:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761395.162969:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565669504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761395.162983:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761395.162988:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761395.162989:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761395.162992:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1330 00000100:00080000:1.0:1789761395.162996:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761395.163001:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761395.163008:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789761395.163013:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761395.163022:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565669632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761395.163070:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761395/real 1789761395] req@ffff93f844ccb480 x1876699565669248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761395.163087:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565669248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761395.163096:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761395.163100:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761395.163103:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565669248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761395.163130:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565669376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761395.163145:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761395.163154:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761395/real 1789761395] req@ffff93f844ccaa00 x1876699565669376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761395.163163:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565669376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761395.163170:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761395.163172:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761395.163174:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565669376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761395.163178:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565669504:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761395.163189:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761395.163195:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761395/real 1789761395] req@ffff93f844cc8000 x1876699565669504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761395.163202:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565669504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761395.163208:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761395.163210:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761395.163212:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565669504:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761395.163215:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565669632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761395.163226:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761395.163231:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761395/real 1789761395] req@ffff93f844ccb800 x1876699565669632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761395.163238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565669632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761395.163243:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761395.163245:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761395.163248:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565669632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761395.226805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761396.250822:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761397.274798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761398.298816:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761399.322807:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761400.282749:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761400.282760:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761400.282765:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761400.282776:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1335 00000100:00080000:1.0:1789761400.282786:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761400.282792:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761400.282825:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565669760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761400.282849:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761400.282852:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761400.282853:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761400.282870:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1335 00000100:00080000:1.0:1789761400.282877:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761400.282882:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761400.282893:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761400.282900:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565669760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761400.282914:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565669888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761400.282930:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761400.282934:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761400.282936:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761400.282941:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1335 00000100:00080000:1.0:1789761400.282947:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761400.282953:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761400.282959:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761400.282970:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565670016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761400.282980:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789761400.282992:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761400.282995:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761400.282997:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761400.283002:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1335 00000100:00080000:1.0:1789761400.283009:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761400.283012:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761400.283019:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761400.283033:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565670144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761400.283040:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761400/real 1789761400] req@ffff93f844ccb100 x1876699565669760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761400.283067:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565669760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761400.283081:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761400.283087:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761400.283091:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565669760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761400.283124:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565669888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761400.283155:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761400.283171:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761400/real 1789761400] req@ffff93f844cc9c00 x1876699565669888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761400.283185:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565669888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761400.283194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761400.283198:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761400.283203:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565669888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761400.283211:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565670016:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761400.283232:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761400.283241:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761400/real 1789761400] req@ffff93f844cc8e00 x1876699565670016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761400.283250:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565670016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761400.283256:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761400.283258:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761400.283260:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565670016:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761400.283264:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565670144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761400.283277:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761400.283283:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761400/real 1789761400] req@ffff93f844cc9500 x1876699565670144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761400.283291:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565670144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761400.283296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761400.283298:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761400.283300:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565670144:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761400.346806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761401.370833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761402.394803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761403.418798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761404.442846:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761405.402732:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761405.402740:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761405.402744:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761405.402753:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1340 00000100:00080000:1.0:1789761405.402765:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761405.402773:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761405.402813:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565670272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761405.402854:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761405.402860:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761405.402862:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761405.402866:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1340 00000100:00080000:1.0:1789761405.402871:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761405.402876:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761405.402887:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761405.402903:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565670272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761405.402906:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565670400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761405.402921:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761405.402925:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761405.402927:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761405.402932:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1340 00000100:00080000:1.0:1789761405.402937:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761405.402941:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761405.402947:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761405.402961:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565670528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761405.402972:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789761405.402976:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761405.402981:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761405.402983:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761405.402989:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1340 00000100:00080000:1.0:1789761405.402994:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761405.402999:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761405.403005:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761405.403017:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565670656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761405.403030:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761405/real 1789761405] req@ffff93f844ccad80 x1876699565670272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761405.403051:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565670272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761405.403062:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761405.403067:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761405.403071:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565670272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761405.403095:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565670400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761405.403118:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761405.403131:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761405/real 1789761405] req@ffff93f844cca680 x1876699565670400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761405.403143:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565670400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761405.403151:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761405.403155:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761405.403158:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565670400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761405.403165:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565670528:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761405.403185:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761405.403196:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761405/real 1789761405] req@ffff93f844cc9500 x1876699565670528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761405.403208:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565670528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761405.403217:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761405.403220:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761405.403223:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565670528:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761405.403229:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565670656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761405.403247:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761405.403256:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761405/real 1789761405] req@ffff93f844cc8e00 x1876699565670656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761405.403267:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565670656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761405.403275:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761405.403279:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761405.403282:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565670656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761405.466816:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761406.490790:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761407.514831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761408.538809:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761409.562907:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761410.522778:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761410.522804:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761410.522809:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761410.522820:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1345 00000100:00080000:0.0:1789761410.522831:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761410.522839:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761410.522881:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565670784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761410.522907:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761410.522911:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761410.522912:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761410.522915:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1345 00000100:00080000:0.0:1789761410.522918:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761410.522920:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761410.522929:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761410.522939:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565670912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761410.522950:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761410.522954:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761410.522957:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761410.522962:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1345 00000100:00080000:0.0:1789761410.522967:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761410.522972:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761410.522978:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761410.522989:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565670784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761410.522995:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565671040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761410.523015:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761410.523020:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761410.523023:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761410.523029:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1345 00000100:00080000:0.0:1789761410.523035:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761410.523040:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761410.523047:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761410.523064:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565671168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761410.523076:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761410.523137:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761410/real 1789761410] req@ffff93f850d43100 x1876699565670784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761410.523160:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565670784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761410.523171:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761410.523176:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761410.523181:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565670784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761410.523189:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565670912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761410.523212:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761410.523224:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761410/real 1789761410] req@ffff93f850d41180 x1876699565670912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761410.523238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565670912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761410.523247:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761410.523251:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761410.523255:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565670912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761410.523280:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565671040:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761410.523300:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761410.523311:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761410/real 1789761410] req@ffff93f850d40000 x1876699565671040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761410.523332:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565671040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761410.523341:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761410.523345:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761410.523349:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565671040:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761410.523356:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565671168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761410.523376:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761410.523386:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761410/real 1789761410] req@ffff93f850d42680 x1876699565671168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761410.523399:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565671168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761410.523408:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761410.523412:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761410.523416:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565671168:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761410.586843:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761411.610805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761412.634805:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761413.658829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761414.682771:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761415.642781:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761415.642796:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761415.642802:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761415.642814:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1350 00000100:00080000:1.0:1789761415.642826:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761415.642835:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761415.642888:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565671296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761415.642918:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761415.642922:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761415.642923:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761415.642928:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1350 00000100:00080000:1.0:1789761415.642932:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761415.642936:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761415.642945:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761415.642952:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565671296:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761415.642962:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565671424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761415.642976:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761415.642980:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761415.642982:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761415.642987:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1350 00000100:00080000:1.0:1789761415.642998:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761415.643004:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761415.643010:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789761415.643025:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761415.643027:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565671552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761415.643042:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761415.643047:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761415.643049:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761415.643054:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1350 00000100:00080000:1.0:1789761415.643059:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761415.643063:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761415.643070:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761415.643084:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565671680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761415.643087:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761415/real 1789761415] req@ffff93f844cc9c00 x1876699565671296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761415.643120:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565671296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761415.643134:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761415.643139:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761415.643143:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565671296:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761415.643173:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565671424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761415.643194:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761415.643204:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761415/real 1789761415] req@ffff93f844ccb100 x1876699565671424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761415.643213:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565671424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761415.643220:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761415.643222:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761415.643224:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565671424:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761415.643228:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565671552:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761415.643242:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761415.643248:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761415/real 1789761415] req@ffff93f844ccb800 x1876699565671552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761415.643256:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565671552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761415.643272:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761415.643276:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761415.643280:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565671552:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761415.643286:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565671680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761415.643309:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761415.643319:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761415/real 1789761415] req@ffff93f844cc8000 x1876699565671680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761415.643327:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565671680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761415.643333:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761415.643336:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761415.643338:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565671680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761415.706826:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761416.730821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761417.754829:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761418.778800:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761419.802845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761420.762798:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761420.762807:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761420.762813:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761420.762827:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1355 00000100:00080000:2.0:1789761420.762840:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761420.762850:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761420.762892:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565671808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761420.762921:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761420.762925:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761420.762926:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761420.762930:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1355 00000100:00080000:2.0:1789761420.762934:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761420.762938:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761420.762947:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761420.762963:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565671808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761420.762965:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565671936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761420.762984:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761420.762988:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761420.762991:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761420.762997:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1355 00000100:00080000:2.0:1789761420.763002:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761420.763007:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761420.763015:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761420.763030:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565672064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761420.763041:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761420.763046:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761420.763049:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1789761420.763052:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761420.763057:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1355 00000100:00080000:2.0:1789761420.763062:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761420.763068:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761420.763075:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761420.763088:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565672192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761420.763118:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761420/real 1789761420] req@ffff93f8517c2300 x1876699565671808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761420.763136:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565671808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761420.763146:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761420.763150:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761420.763153:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565671808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761420.763179:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565671936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761420.763197:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761420.763207:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761420/real 1789761420] req@ffff93f8517c1f80 x1876699565671936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761420.763215:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565671936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761420.763221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761420.763224:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761420.763226:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565671936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761420.763230:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565672064:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761420.763242:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761420.763248:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761420/real 1789761420] req@ffff93f8517c3b80 x1876699565672064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761420.763256:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565672064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761420.763262:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761420.763263:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761420.763265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565672064:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761420.763270:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565672192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761420.763283:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761420.763289:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761420/real 1789761420] req@ffff93f8517c0000 x1876699565672192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761420.763297:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565672192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761420.763303:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761420.763306:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761420.763309:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565672192:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761420.826798:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761421.850864:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761422.874829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761423.898846:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761424.922788:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761425.882801:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761425.882812:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761425.882818:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761425.882831:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1360 00000100:00080000:2.0:1789761425.882842:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761425.882851:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761425.882897:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565672320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761425.882920:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761425.882923:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761425.882925:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761425.882928:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1360 00000100:00080000:2.0:1789761425.882931:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761425.882934:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761425.882942:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761425.882953:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565672448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761425.882963:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761425.882966:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761425.882967:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761425.882970:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1360 00000100:00080000:2.0:1789761425.882972:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761425.882975:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761425.882978:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761425.882987:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565672320:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761425.882989:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565672576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761425.883010:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761425.883012:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761425.883014:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761425.883017:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1360 00000100:00080000:2.0:1789761425.883019:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761425.883022:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761425.883025:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761425.883039:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565672704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761425.883085:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761425.883140:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761425/real 1789761425] req@ffff93f8517c0000 x1876699565672320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761425.883160:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565672320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761425.883172:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761425.883177:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761425.883181:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565672320:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761425.883190:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565672448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761425.883212:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761425.883224:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761425/real 1789761425] req@ffff93f8517c3b80 x1876699565672448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761425.883236:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565672448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761425.883250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761425.883253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761425.883257:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565672448:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761425.883282:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565672576:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761425.883301:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761425.883312:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761425/real 1789761425] req@ffff93f8517c1f80 x1876699565672576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761425.883323:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565672576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761425.883331:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761425.883334:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761425.883337:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565672576:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761425.883343:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565672704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761425.883359:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761425.883370:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761425/real 1789761425] req@ffff93f8517c2300 x1876699565672704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761425.883381:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565672704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761425.883388:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761425.883391:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761425.883395:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565672704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761425.946844:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761426.970830:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761427.994782:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761429.018839:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761430.042791:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761431.002830:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761431.002842:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761431.002996:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761431.003011:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1365 00000100:00080000:1.0:1789761431.003024:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761431.003033:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761431.003073:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565672832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761431.003105:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761431.003109:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761431.003111:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761431.003115:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1365 00000100:00080000:1.0:1789761431.003119:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761431.003123:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761431.003133:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761431.003146:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565672960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761431.003158:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761431.003161:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761431.003163:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761431.003167:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1365 00000100:00080000:1.0:1789761431.003170:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761431.003173:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761431.003177:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761431.003185:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565673088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761431.003192:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761431.003194:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761431.003195:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761431.003198:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1365 00000100:00080000:1.0:1789761431.003199:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761431.003201:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761431.003203:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761431.003209:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565673216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761431.003273:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565672832:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1789761431.003357:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761431.003419:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761431/real 1789761431] req@ffff93f844ccaa00 x1876699565672832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761431.003443:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565672832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761431.003458:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761431.003464:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761431.003468:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565672832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761431.003478:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565672960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761431.003504:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761431.003519:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761431/real 1789761431] req@ffff93f844ccb480 x1876699565672960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761431.003532:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565672960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761431.003541:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761431.003545:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761431.003549:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565672960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761431.003557:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565673088:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761431.003595:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761431.003610:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761431/real 1789761431] req@ffff93f844cc8700 x1876699565673088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761431.003624:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565673088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761431.003690:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761431.003695:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761431.003699:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565673088:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761431.003706:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565673216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761431.003731:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761431.003745:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761431/real 1789761431] req@ffff93f844ccbb80 x1876699565673216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761431.003759:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565673216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761431.003768:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761431.003771:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761431.003775:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565673216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761431.066877:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761432.090765:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761433.114861:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761434.138829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761435.162790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761436.122741:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761436.122749:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761436.122753:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761436.122761:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1371 00000100:00080000:0.0:1789761436.122769:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761436.122775:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761436.122808:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565673344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761436.122832:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761436.122835:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761436.122837:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761436.122839:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1371 00000100:00080000:0.0:1789761436.122842:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761436.122844:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761436.122852:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761436.122861:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565673472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761436.122869:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761436.122872:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761436.122873:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761436.122876:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1371 00000100:00080000:0.0:1789761436.122878:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761436.122883:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761436.122886:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565673344:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1789761436.122887:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761436.122896:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565673600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761436.122909:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761436.122911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761436.122912:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761436.122915:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1371 00000100:00080000:0.0:1789761436.122917:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761436.122919:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761436.122923:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761436.122931:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565673728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761436.122949:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761436.122982:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761436/real 1789761436] req@ffff93f850d41c00 x1876699565673344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761436.122994:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565673344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761436.123002:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761436.123005:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761436.123008:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565673344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761436.123015:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565673472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761436.123030:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761436.123044:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761436/real 1789761436] req@ffff93f850d40e00 x1876699565673472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761436.123052:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565673472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761436.123058:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761436.123060:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761436.123062:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565673472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761436.123080:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565673600:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761436.123093:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761436.123099:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761436/real 1789761436] req@ffff93f850d40700 x1876699565673600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761436.123107:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565673600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761436.123112:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761436.123114:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761436.123117:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565673600:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761436.123121:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565673728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761436.123140:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761436.123152:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761436/real 1789761436] req@ffff93f850d43b80 x1876699565673728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761436.123165:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565673728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761436.123174:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761436.123177:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761436.123180:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565673728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761436.186754:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761437.210806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761438.234784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761439.258800:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761440.282755:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761441.242738:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761441.242745:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761441.242749:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761441.242756:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1376 00000100:00080000:3.0:1789761441.242764:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761441.242770:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761441.242802:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565673856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761441.242824:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761441.242828:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761441.242830:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761441.242833:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1376 00000100:00080000:3.0:1789761441.242837:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761441.242842:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761441.242852:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761441.242866:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565673856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761441.242870:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565673984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761441.242883:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761441.242886:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761441.242888:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761441.242891:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1376 00000100:00080000:3.0:1789761441.242896:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761441.242899:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761441.242903:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761441.242914:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565674112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761441.242921:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761441.242924:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:1.0:1789761441.242927:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761441.242928:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761441.242935:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1376 00000100:00080000:3.0:1789761441.242938:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761441.242941:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761441.242945:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761441.242952:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565674240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761441.242977:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761441/real 1789761441] req@ffff93f844d67b80 x1876699565673856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761441.242992:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565673856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761441.243000:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761441.243003:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761441.243006:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565673856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761441.243030:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565673984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761441.243047:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761441.243056:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761441/real 1789761441] req@ffff93f844d64700 x1876699565673984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761441.243064:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565673984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761441.243074:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761441.243076:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761441.243079:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565673984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761441.243082:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565674112:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761441.243093:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761441.243099:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761441/real 1789761441] req@ffff93f844d66680 x1876699565674112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761441.243106:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565674112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761441.243111:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761441.243113:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761441.243115:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565674112:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761441.243118:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565674240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761441.243128:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761441.243134:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761441/real 1789761441] req@ffff93f844d65880 x1876699565674240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761441.243141:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565674240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761441.243146:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761441.243148:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761441.243150:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565674240:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761441.306759:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761442.330796:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761443.354810:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761444.378813:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761445.402797:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761446.362748:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761446.362756:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761446.362760:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761446.362769:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1381 00000100:00080000:3.0:1789761446.362777:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761446.362784:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761446.362821:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565674368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761446.362846:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761446.362849:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761446.362850:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761446.362853:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1381 00000100:00080000:3.0:1789761446.362855:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761446.362858:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761446.362870:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761446.362880:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565674496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761446.362887:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565674368:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761446.362890:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761446.362895:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761446.362896:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761446.362900:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1381 00000100:00080000:3.0:1789761446.362902:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761446.362904:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761446.362908:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761446.362917:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565674624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761446.362924:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761446.362927:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761446.362928:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761446.362931:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1381 00000100:00080000:3.0:1789761446.362940:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761446.362943:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761446.362946:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761446.362951:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761446.362956:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565674752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761446.362997:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761446/real 1789761446] req@ffff93f844d67480 x1876699565674368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761446.363011:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565674368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761446.363018:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761446.363022:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761446.363027:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565674368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761446.363047:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565674496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761446.363063:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761446.363071:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761446/real 1789761446] req@ffff93f844d67800 x1876699565674496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761446.363080:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565674496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761446.363085:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761446.363088:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761446.363090:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565674496:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761446.363093:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565674624:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761446.363107:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761446.363122:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761446/real 1789761446] req@ffff93f844d67100 x1876699565674624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761446.363130:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565674624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761446.363136:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761446.363138:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761446.363141:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565674624:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761446.363146:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565674752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761446.363159:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761446.363165:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761446/real 1789761446] req@ffff93f844d66a00 x1876699565674752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761446.363172:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565674752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761446.363177:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761446.363179:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761446.363181:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565674752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761446.426747:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761447.450815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761448.474830:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761449.498791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761450.522829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761451.482747:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761451.482758:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761451.482762:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761451.482773:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1386 00000100:00080000:1.0:1789761451.482784:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761451.482792:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761451.482831:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565674880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761451.482857:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761451.482862:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761451.482864:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761451.482869:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1386 00000100:00080000:1.0:1789761451.482874:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761451.482879:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761451.482892:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761451.482903:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565674880:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761451.482914:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565675008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761451.482927:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761451.482931:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761451.482933:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761451.482939:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1386 00000100:00080000:1.0:1789761451.482946:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761451.482952:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761451.482960:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761451.482974:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565675136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761451.482984:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761451.482989:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761451.482992:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761451.482998:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1386 00000100:00080000:1.0:1789761451.483004:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761451.483009:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761451.483016:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761451.483026:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565675008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761451.483036:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565675264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761451.483060:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565675136:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761451.483089:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565675264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761451.546798:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761452.570813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761453.594754:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789761454.554949:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789761454.554984:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789761454.554999:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761454.555036:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761454.555048:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761454.555058:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789761454.555111:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761451/real 1789761454] req@ffff93f844cc9880 x1876699565675264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761454.555151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565675264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761454.555166:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761454.555172:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761454.555178:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565675264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761454.555189:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761451/real 1789761454] req@ffff93f844cc8380 x1876699565675008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761454.555201:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565675008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761454.555210:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761454.555215:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761454.555218:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565675008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761454.555226:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761451/real 1789761454] req@ffff93f844cc9180 x1876699565675136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761454.555239:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565675136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761454.555247:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761454.555252:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761454.555256:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565675136:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761454.555265:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761451/real 1789761454] req@ffff93f844cc8a80 x1876699565674880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761454.555289:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565674880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761454.555298:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761454.555302:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761454.555305:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565674880:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761454.618877:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761455.642829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761456.602769:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761456.602780:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761456.602785:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761456.602797:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1391 00000100:00080000:2.0:1789761456.602807:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761456.602815:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761456.602862:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565675392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761456.602893:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761456.602898:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761456.602901:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761456.602906:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1391 00000100:00080000:2.0:1789761456.602921:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761456.602927:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761456.602940:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761456.602961:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565675520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761456.602974:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761456.602978:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761456.602980:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761456.602984:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1391 00000100:00080000:2.0:1789761456.602989:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1789761456.602997:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565675392:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761456.602999:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761456.603014:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761456.603040:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565675648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761456.603055:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761456.603061:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761456.603064:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761456.603069:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1391 00000100:00080000:2.0:1789761456.603074:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761456.603080:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761456.603087:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789761456.603097:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761456.603107:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565675776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761456.603159:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761456/real 1789761456] req@ffff93f8517c2680 x1876699565675392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761456.603188:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565675392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761456.603207:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761456.603214:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761456.603223:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565675392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761456.603243:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565675520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761456.603306:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761456.603334:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761456/real 1789761456] req@ffff93f8517c0380 x1876699565675520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761456.603357:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565675520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761456.603367:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761456.603372:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761456.603375:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565675520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761456.603407:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565675648:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761456.603429:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761456.603454:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761456/real 1789761456] req@ffff93f8517c1500 x1876699565675648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761456.603468:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565675648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761456.603482:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761456.603486:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761456.603490:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565675648:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761456.603497:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565675776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761456.603519:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761456.603532:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761456/real 1789761456] req@ffff93f8517c0a80 x1876699565675776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761456.603545:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565675776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761456.603554:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761456.603558:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761456.603561:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565675776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761456.666829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761457.690868:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761458.714848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761459.738796:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761460.762802:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761461.722786:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761461.722796:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761461.722801:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761461.722812:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1396 00000100:00080000:3.0:1789761461.722823:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761461.722830:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761461.722871:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565675904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761461.722897:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761461.722900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761461.722901:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761461.722905:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1396 00000100:00080000:3.0:1789761461.722907:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761461.722909:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761461.722923:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761461.722934:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565676032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761461.722942:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761461.722944:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761461.722946:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761461.722948:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1396 00000100:00080000:3.0:1789761461.722950:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761461.722953:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761461.722958:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761461.722960:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565675904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761461.722968:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565676160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761461.722977:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761461.722980:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761461.722981:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761461.722983:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1396 00000100:00080000:3.0:1789761461.722987:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761461.722989:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761461.722992:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761461.722999:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565676288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761461.723041:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761461.723092:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761461/real 1789761461] req@ffff93f844d65c00 x1876699565675904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761461.723109:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565675904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761461.723119:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761461.723122:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761461.723125:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565675904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761461.723139:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565676032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761461.723163:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761461.723178:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761461/real 1789761461] req@ffff93f844d64000 x1876699565676032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761461.723199:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565676032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761461.723206:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761461.723209:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761461.723212:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565676032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761461.723245:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565676160:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761461.723271:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761461.723284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761461/real 1789761461] req@ffff93f844d66d80 x1876699565676160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761461.723298:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565676160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761461.723308:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761461.723312:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761461.723315:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565676160:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761461.723332:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565676288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761461.723360:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761461.723373:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761461/real 1789761461] req@ffff93f844d64e00 x1876699565676288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761461.723383:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565676288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761461.723389:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761461.723391:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761461.723393:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565676288:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761461.786783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761462.810832:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761463.834803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761464.858827:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761465.882877:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761466.842772:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761466.842787:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761466.842802:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761466.842816:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1401 00000100:00080000:3.0:1789761466.842829:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761466.842839:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761466.842889:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565676416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761466.842920:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761466.842924:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761466.842927:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761466.842932:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1401 00000100:00080000:3.0:1789761466.842937:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761466.842943:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761466.842955:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761466.842956:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565676416:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761466.842978:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565676544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761466.842992:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761466.842996:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761466.842999:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761466.843004:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1401 00000100:00080000:3.0:1789761466.843009:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761466.843014:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761466.843019:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761466.843036:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565676672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761466.843048:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761466.843052:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761466.843057:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761466.843060:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761466.843066:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1401 00000100:00080000:3.0:1789761466.843071:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761466.843077:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761466.843084:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761466.843099:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565676800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761466.843115:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761466/real 1789761466] req@ffff93f844d65500 x1876699565676416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761466.843135:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565676416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761466.843144:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761466.843147:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761466.843151:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565676416:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761466.843176:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565676544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761466.843195:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761466.843204:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761466/real 1789761466] req@ffff93f844d65f80 x1876699565676544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761466.843213:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565676544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761466.843223:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761466.843226:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761466.843228:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565676544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761466.843232:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565676672:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761466.843244:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761466.843250:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761466/real 1789761466] req@ffff93f844d64a80 x1876699565676672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761466.843257:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565676672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761466.843263:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761466.843264:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761466.843266:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565676672:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761466.843269:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565676800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761466.843279:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761466.843285:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761466/real 1789761466] req@ffff93f844d64380 x1876699565676800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761466.843293:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565676800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761466.843298:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761466.843299:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761466.843302:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565676800:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761466.906841:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761467.930851:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761468.954814:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761469.978801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761471.002783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761471.962770:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761471.962784:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761471.962790:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761471.962805:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1406 00000100:00080000:1.0:1789761471.962820:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761471.962831:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761471.962882:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565676928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761471.962915:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761471.962919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761471.962922:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761471.962927:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1406 00000100:00080000:1.0:1789761471.962932:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761471.962937:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761471.962948:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761471.962955:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565676928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761471.962968:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565677056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761471.962979:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761471.962982:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761471.962984:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761471.962987:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1406 00000100:00080000:1.0:1789761471.962990:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761471.962994:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761471.962999:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761471.963011:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565677184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761471.963019:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761471.963022:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761471.963023:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761471.963026:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1406 00000100:00080000:1.0:1789761471.963028:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761471.963030:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761471.963033:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761471.963034:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761471.963044:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565677312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761471.963101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761471/real 1789761471] req@ffff93f844cca300 x1876699565676928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761471.963124:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565676928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761471.963136:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761471.963140:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761471.963144:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565676928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761471.963176:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565677056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761471.963204:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761471.963217:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761471/real 1789761471] req@ffff93f844cc9f80 x1876699565677056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761471.963227:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565677056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761471.963234:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761471.963236:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761471.963238:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565677056:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761471.963243:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565677184:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761471.963259:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761471.963266:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761471/real 1789761471] req@ffff93f844cc8a80 x1876699565677184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761471.963275:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565677184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761471.963285:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761471.963288:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761471.963290:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565677184:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761471.963293:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565677312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761471.963304:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761471.963310:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761471/real 1789761471] req@ffff93f844cc9180 x1876699565677312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761471.963317:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565677312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761471.963338:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761471.963341:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761471.963343:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565677312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761472.026843:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761473.050811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761474.074871:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761475.098798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761476.122840:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761477.082773:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761477.082787:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761477.082793:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761477.082806:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1412 00000100:00080000:3.0:1789761477.082819:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761477.082827:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761477.082868:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565677440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761477.082895:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761477.082899:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761477.082902:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761477.082906:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1412 00000100:00080000:3.0:1789761477.082911:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761477.082916:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761477.082927:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761477.082934:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565677440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761477.082940:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565677568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761477.082951:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761477.082954:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761477.082957:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761477.082960:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1412 00000100:00080000:3.0:1789761477.082963:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761477.082967:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761477.082972:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761477.082983:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565677696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761477.082993:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761477.082996:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761477.082998:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761477.083002:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1412 00000100:00080000:3.0:1789761477.083005:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761477.083009:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1789761477.083010:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1789761477.083014:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761477.083024:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565677824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761477.083075:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761477/real 1789761477] req@ffff93f844d65180 x1876699565677440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761477.083094:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565677440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761477.083103:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761477.083107:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761477.083110:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565677440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761477.083135:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565677568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761477.083163:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761477.083173:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761477/real 1789761477] req@ffff93f844d66300 x1876699565677568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761477.083182:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565677568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761477.083188:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761477.083190:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761477.083192:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565677568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761477.083196:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565677696:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761477.083208:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761477.083214:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761477/real 1789761477] req@ffff93f844d64380 x1876699565677696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761477.083222:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565677696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761477.083228:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761477.083230:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761477.083232:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565677696:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761477.083241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565677824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761477.083253:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761477.083260:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761477/real 1789761477] req@ffff93f844d64a80 x1876699565677824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761477.083268:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565677824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761477.083274:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761477.083275:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761477.083277:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565677824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761477.146854:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761478.170853:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761479.194873:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761480.218825:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761481.242834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761482.202750:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761482.202765:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761482.202772:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761482.202785:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1417 00000100:00080000:3.0:1789761482.202799:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761482.202809:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761482.202858:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565677952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761482.202889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761482.202893:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761482.202896:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761482.202901:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1417 00000100:00080000:3.0:1789761482.202906:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761482.202913:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761482.202925:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761482.202936:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565677952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761482.202944:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565678080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761482.202959:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761482.202963:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761482.202966:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761482.202972:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1417 00000100:00080000:3.0:1789761482.202995:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761482.203001:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1789761482.203007:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1789761482.203010:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761482.203029:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565678208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761482.203042:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761482.203047:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761482.203049:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761482.203054:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1417 00000100:00080000:3.0:1789761482.203059:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761482.203064:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761482.203070:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761482.203071:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761482/real 1789761482] req@ffff93f844d65f80 x1876699565677952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761482.203087:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565678336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761482.203098:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565677952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761482.203114:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761482.203119:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761482.203125:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565677952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761482.203156:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565678080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761482.203180:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761482.203191:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761482/real 1789761482] req@ffff93f844d65500 x1876699565678080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761482.203200:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565678080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761482.203211:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761482.203214:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761482.203216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565678080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761482.203220:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565678208:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761482.203234:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761482.203240:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761482/real 1789761482] req@ffff93f844d64e00 x1876699565678208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761482.203262:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565678208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761482.203268:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761482.203270:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761482.203273:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565678208:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761482.203277:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565678336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761482.203292:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761482.203299:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761482/real 1789761482] req@ffff93f844d66d80 x1876699565678336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761482.203307:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565678336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761482.203313:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761482.203315:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761482.203318:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565678336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761482.266775:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761483.290817:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761484.314797:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761485.338805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761486.362799:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761487.322729:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761487.322740:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761487.322744:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761487.322756:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1422 00000100:00080000:3.0:1789761487.322767:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761487.322775:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761487.322817:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565678464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761487.322840:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761487.322843:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761487.322844:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761487.322848:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1422 00000100:00080000:3.0:1789761487.322851:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761487.322855:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761487.322864:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761487.322874:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565678592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761487.322879:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565678464:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761487.322886:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761487.322889:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761487.322891:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761487.322894:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1422 00000100:00080000:3.0:1789761487.322896:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761487.322898:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761487.322902:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761487.322913:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565678720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761487.322921:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761487.322924:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761487.322925:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761487.322928:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1422 00000100:00080000:3.0:1789761487.322938:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761487.322941:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761487.322944:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761487.322955:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761487.322955:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565678848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761487.323010:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761487/real 1789761487] req@ffff93f844d64000 x1876699565678464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761487.323029:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565678464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761487.323038:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761487.323042:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761487.323046:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565678464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761487.323073:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565678592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761487.323091:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761487.323100:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761487/real 1789761487] req@ffff93f844d65c00 x1876699565678592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761487.323109:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565678592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761487.323115:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761487.323117:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761487.323119:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565678592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761487.323123:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565678720:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761487.323135:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761487.323142:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761487/real 1789761487] req@ffff93f844d66a00 x1876699565678720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761487.323150:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565678720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761487.323155:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761487.323157:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761487.323159:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565678720:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761487.323162:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565678848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761487.323173:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761487.323179:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761487/real 1789761487] req@ffff93f844d67100 x1876699565678848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761487.323187:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565678848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761487.323192:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761487.323194:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761487.323196:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565678848:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761487.386871:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761488.410780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761489.434841:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761490.458834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761491.482804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761492.442738:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761492.442750:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761492.442755:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761492.442766:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1427 00000100:00080000:3.0:1789761492.442777:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761492.442785:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761492.442826:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565678976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761492.442852:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761492.442855:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761492.442857:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761492.442860:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1427 00000100:00080000:3.0:1789761492.442862:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761492.442865:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761492.442874:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761492.442885:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565679104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761492.442894:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761492.442897:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761492.442899:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761492.442905:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1427 00000100:00080000:3.0:1789761492.442910:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789761492.442911:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565678976:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761492.442915:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761492.442928:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761492.442937:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565679232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761492.442945:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761492.442949:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761492.442950:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761492.442957:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1427 00000100:00080000:3.0:1789761492.442959:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761492.442962:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761492.442966:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761492.442976:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565679360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761492.443003:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761492.443066:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761492/real 1789761492] req@ffff93f844d67800 x1876699565678976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761492.443088:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565678976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761492.443100:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761492.443106:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761492.443110:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565678976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761492.443134:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565679104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761492.443155:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761492.443182:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761492/real 1789761492] req@ffff93f844d67480 x1876699565679104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761492.443195:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565679104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761492.443208:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761492.443211:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761492.443217:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565679104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761492.443229:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565679232:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761492.443263:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761492.443281:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761492/real 1789761492] req@ffff93f844d65880 x1876699565679232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761492.443293:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565679232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761492.443301:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761492.443303:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761492.443306:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565679232:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761492.443314:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565679360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761492.443336:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761492.443344:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761492/real 1789761492] req@ffff93f844d66680 x1876699565679360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761492.443352:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565679360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761492.443357:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761492.443359:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761492.443361:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565679360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761492.506803:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761493.530852:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761494.554802:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761495.578821:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761496.602777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761497.563091:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761497.563100:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761497.563104:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761497.563112:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1432 00000100:00080000:0.0:1789761497.563120:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761497.563127:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761497.563175:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565679488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761497.563202:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761497.563206:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761497.563209:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761497.563213:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1432 00000100:00080000:0.0:1789761497.563218:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761497.563224:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761497.563236:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761497.563250:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565679488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761497.563251:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565679616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761497.563266:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761497.563269:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761497.563270:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761497.563273:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1432 00000100:00080000:0.0:1789761497.563276:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761497.563279:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761497.563283:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761497.563291:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565679744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761497.563299:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761497.563301:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761497.563302:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761497.563305:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1432 00000100:00080000:0.0:1789761497.563307:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761497.563309:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761497.563312:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761497.563319:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565679872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761497.563333:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761497.563377:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761497/real 1789761497] req@ffff93f850d42300 x1876699565679488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761497.563391:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565679488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761497.563399:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761497.563402:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761497.563405:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565679488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761497.563424:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565679616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761497.563446:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761497.563459:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761497/real 1789761497] req@ffff93f850d40380 x1876699565679616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761497.563473:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565679616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761497.563483:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761497.563487:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761497.563490:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565679616:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761497.563497:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565679744:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761497.563514:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761497.563523:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761497/real 1789761497] req@ffff93f850d43b80 x1876699565679744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761497.563531:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565679744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761497.563536:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761497.563539:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761497.563541:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565679744:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761497.563545:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565679872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761497.563555:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761497.563561:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761497/real 1789761497] req@ffff93f850d40700 x1876699565679872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761497.563571:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565679872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761497.563585:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761497.563589:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761497.563592:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565679872:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761497.626795:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761498.650806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761499.674825:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761500.698810:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761501.722801:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761502.682758:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761502.682770:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761502.682775:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761502.682786:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1437 00000100:00080000:0.0:1789761502.682797:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761502.682804:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761502.682854:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565680000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761502.682887:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761502.682892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761502.682894:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761502.682901:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1437 00000100:00080000:0.0:1789761502.682906:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761502.682911:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761502.682921:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761502.682929:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565680000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761502.682939:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565680128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761502.682949:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761502.682952:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761502.682953:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761502.682958:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1437 00000100:00080000:0.0:1789761502.682961:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761502.682964:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761502.682969:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761502.682981:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565680256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761502.682994:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761502.682997:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761502.682999:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761502.683004:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1437 00000100:00080000:0.0:1789761502.683007:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761502.683009:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1789761502.683011:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1789761502.683014:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761502.683026:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565680384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761502.683062:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761502/real 1789761502] req@ffff93f850d40e00 x1876699565680000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761502.683077:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565680000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761502.683085:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761502.683090:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761502.683112:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565680000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761502.683139:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565680128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761502.683177:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761502.683192:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761502/real 1789761502] req@ffff93f850d41c00 x1876699565680128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761502.683205:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565680128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761502.683211:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761502.683214:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761502.683216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565680128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761502.683222:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565680256:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761502.683238:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761502.683246:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761502/real 1789761502] req@ffff93f850d42680 x1876699565680256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761502.683254:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565680256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761502.683260:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761502.683262:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761502.683264:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565680256:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761502.683268:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565680384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761502.683279:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761502.683285:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761502/real 1789761502] req@ffff93f850d40000 x1876699565680384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761502.683293:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565680384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761502.683299:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761502.683303:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761502.683306:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565680384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761502.746808:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761503.770763:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761504.794840:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761505.818770:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761506.842887:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761507.802784:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761507.802795:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761507.802800:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761507.802813:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1442 00000100:00080000:2.0:1789761507.802826:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761507.802835:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761507.802888:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565680512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761507.802919:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761507.802922:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761507.802924:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761507.802928:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1442 00000100:00080000:2.0:1789761507.802932:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761507.802936:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761507.802945:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761507.802948:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565680512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761507.802960:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565680640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761507.802969:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761507.802972:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761507.802974:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761507.802976:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1442 00000100:00080000:2.0:1789761507.802979:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761507.802981:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761507.802985:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761507.802993:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565680768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761507.803013:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761507.803016:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761507.803017:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761507.803020:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1442 00000100:00080000:2.0:1789761507.803024:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761507.803026:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761507.803029:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761507.803040:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565680896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761507.803041:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761507.803113:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761507/real 1789761507] req@ffff93f8517c1180 x1876699565680512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761507.803133:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565680512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761507.803143:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761507.803146:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761507.803150:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565680512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761507.803175:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565680640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761507.803195:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761507.803205:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761507/real 1789761507] req@ffff93f8517c3100 x1876699565680640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761507.803213:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565680640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761507.803219:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761507.803222:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761507.803224:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565680640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761507.803228:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565680768:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761507.803242:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761507.803248:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761507/real 1789761507] req@ffff93f8517c2a00 x1876699565680768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761507.803256:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565680768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761507.803262:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761507.803264:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761507.803265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565680768:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761507.803269:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565680896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761507.803281:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761507.803287:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761507/real 1789761507] req@ffff93f8517c2d80 x1876699565680896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761507.803295:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565680896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761507.803300:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761507.803302:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761507.803304:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565680896:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761507.866797:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761508.890866:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761509.914753:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761510.938858:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761511.962790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761512.922761:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761512.922770:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761512.922775:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761512.922787:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1447 00000100:00080000:1.0:1789761512.922800:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761512.922809:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761512.922846:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565681024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761512.922873:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761512.922876:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761512.922878:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761512.922881:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1447 00000100:00080000:1.0:1789761512.922884:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761512.922886:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761512.922894:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761512.922905:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565681152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761512.922925:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789761512.922932:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565681024:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761512.922936:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761512.922945:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761512.922953:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1447 00000100:00080000:1.0:1789761512.922957:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761512.922962:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761512.922970:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761512.922986:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565681280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761512.922995:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761512.922999:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761512.923002:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761512.923005:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1447 00000100:00080000:1.0:1789761512.923009:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761512.923013:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761512.923032:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761512.923046:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565681408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761512.923046:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761512.923125:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761512/real 1789761512] req@ffff93f844cc8380 x1876699565681024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761512.923148:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565681024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761512.923158:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761512.923162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761512.923165:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565681024:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761512.923194:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565681152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761512.923214:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761512.923224:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761512/real 1789761512] req@ffff93f844cc9880 x1876699565681152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761512.923234:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565681152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761512.923240:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761512.923242:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761512.923245:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565681152:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761512.923249:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565681280:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761512.923261:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761512.923268:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761512/real 1789761512] req@ffff93f844ccbb80 x1876699565681280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761512.923276:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565681280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761512.923282:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761512.923284:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761512.923286:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565681280:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761512.923297:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565681408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761512.923310:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761512.923317:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761512/real 1789761512] req@ffff93f844cc8700 x1876699565681408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761512.923325:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565681408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761512.923335:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761512.923337:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761512.923340:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565681408:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761512.986876:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761514.010778:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761515.034837:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761516.058761:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761517.082798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761518.042772:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761518.042783:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761518.042789:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761518.042802:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1453 00000100:00080000:1.0:1789761518.042814:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761518.042822:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761518.042867:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565681536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761518.042892:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761518.042895:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761518.042897:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761518.042900:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1453 00000100:00080000:1.0:1789761518.042903:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761518.042905:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761518.042919:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761518.042931:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565681664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761518.042934:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565681536:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761518.042942:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761518.042946:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761518.042948:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761518.042952:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1453 00000100:00080000:1.0:1789761518.042954:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761518.042956:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761518.042960:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761518.042969:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565681792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761518.042990:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761518.042994:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761518.042996:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761518.042999:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1453 00000100:00080000:1.0:1789761518.043003:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761518.043006:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761518.043010:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761518.043020:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565681920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761518.043081:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565681664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761518.043110:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565681792:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761518.043126:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565681920:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761518.106792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761519.130817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761520.154811:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789761521.115082:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789761521.115145:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789761521.115163:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761521.115231:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761521.115257:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761521.115282:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789761521.115284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761518/real 1789761521] req@ffff93f844ccaa00 x1876699565681664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761521.115325:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565681664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761521.115341:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761521.115347:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761521.115353:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565681664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761521.115362:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761518/real 1789761521] req@ffff93f844cc8000 x1876699565681792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761521.115385:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565681792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761521.115396:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761521.115402:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761521.115406:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565681792:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761521.115414:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761518/real 1789761521] req@ffff93f844ccb800 x1876699565681920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761521.115427:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565681920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761521.115437:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761521.115446:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761521.115450:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565681920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761521.115467:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761518/real 1789761521] req@ffff93f844ccb480 x1876699565681536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761521.115482:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565681536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761521.115492:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761521.115496:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761521.115499:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565681536:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761521.178790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761522.202820:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761523.162805:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761523.162837:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761523.162843:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761523.162856:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1458 00000100:00080000:2.0:1789761523.162869:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761523.162894:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761523.162957:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565682048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761523.162997:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761523.163002:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761523.163004:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761523.163009:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1458 00000100:00080000:2.0:1789761523.163014:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761523.163018:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761523.163030:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761523.163046:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565682176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761523.163058:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1789761523.163064:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565682048:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761523.163065:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761523.163068:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761523.163077:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1458 00000100:00080000:2.0:1789761523.163083:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761523.163086:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761523.163092:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761523.163107:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565682304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761523.163119:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761523.163123:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761523.163125:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761523.163131:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1458 00000100:00080000:2.0:1789761523.163136:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761523.163140:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761523.163146:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761523.163160:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565682432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761523.163164:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761523.163255:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761523/real 1789761523] req@ffff93f8517c0e00 x1876699565682048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761523.163280:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565682048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761523.163294:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761523.163300:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761523.163305:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565682048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761523.163342:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565682176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761523.163370:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761523.163387:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761523/real 1789761523] req@ffff93f8517c3480 x1876699565682176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761523.163401:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565682176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761523.163411:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761523.163421:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761523.163425:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565682176:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761523.163432:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565682304:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761523.163462:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761523.163475:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761523/real 1789761523] req@ffff93f8517c3800 x1876699565682304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761523.163487:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565682304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761523.163502:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761523.163506:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761523.163509:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565682304:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761523.163516:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565682432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761523.163539:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761523.163551:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761523/real 1789761523] req@ffff93f8517c1880 x1876699565682432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761523.163564:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565682432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761523.163574:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761523.163578:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761523.163582:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565682432:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761523.226809:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761524.250792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761525.274817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761526.298839:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761527.322755:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761528.282808:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761528.282820:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761528.282826:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761528.282839:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1463 00000100:00080000:0.0:1789761528.282851:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761528.282861:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761528.282920:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565682560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761528.282946:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761528.282950:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761528.282953:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761528.282958:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1463 00000100:00080000:0.0:1789761528.282963:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761528.282971:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761528.282982:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761528.282989:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565682560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761528.283001:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565682688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761528.283018:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761528.283024:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761528.283026:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761528.283032:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1463 00000100:00080000:0.0:1789761528.283040:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761528.283046:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761528.283056:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789761528.283063:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761528.283078:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565682816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761528.283095:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1789761528.283097:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761528/real 1789761528] req@ffff93f850d41180 x1876699565682560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761528.283108:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761528.283116:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761528.283131:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1463 00000100:00080000:0.0:1789761528.283136:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761528.283140:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761528.283147:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761528.283163:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565682944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761528.283198:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565682560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761528.283208:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761528.283211:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761528.283214:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565682560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761528.283235:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565682688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761528.283254:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761528.283264:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761528/real 1789761528] req@ffff93f850d43100 x1876699565682688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761528.283272:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565682688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761528.283277:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761528.283279:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761528.283281:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565682688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761528.283285:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565682816:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761528.283297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761528.283302:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761528/real 1789761528] req@ffff93f850d41f80 x1876699565682816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761528.283314:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565682816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761528.283441:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761528.283446:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761528.283452:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565682816:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761528.283461:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565682944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761528.283524:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761528.283555:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761528/real 1789761528] req@ffff93f850d41500 x1876699565682944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761528.283577:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565682944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761528.283586:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761528.283591:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761528.283594:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565682944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761528.346801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761529.370757:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761530.394854:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761531.418769:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761532.442857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761533.402749:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761533.402760:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761533.402765:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761533.402777:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1468 00000100:00080000:0.0:1789761533.402788:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761533.402798:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761533.402856:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565683072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761533.402900:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761533.402903:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761533.402906:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761533.402911:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1468 00000100:00080000:0.0:1789761533.402916:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761533.402921:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761533.402932:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761533.402948:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565683072:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761533.402951:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565683200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761533.402966:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761533.402971:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761533.402974:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761533.402979:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1468 00000100:00080000:0.0:1789761533.402984:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761533.402989:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761533.402996:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761533.403013:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565683328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761533.403031:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789761533.403039:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761533.403045:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761533.403048:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761533.403054:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1468 00000100:00080000:0.0:1789761533.403059:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761533.403064:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761533.403071:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761533.403085:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565683456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761533.403090:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761533/real 1789761533] req@ffff93f850d43800 x1876699565683072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761533.403119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565683072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761533.403132:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761533.403140:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761533.403145:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565683072:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761533.403174:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565683200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761533.403199:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761533.403210:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761533/real 1789761533] req@ffff93f850d43480 x1876699565683200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761533.403219:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565683200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761533.403226:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761533.403228:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761533.403230:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565683200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761533.403234:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565683328:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761533.403248:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761533.403255:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761533/real 1789761533] req@ffff93f850d41500 x1876699565683328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761533.403263:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565683328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761533.403273:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761533.403275:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761533.403278:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565683328:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761533.403281:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565683456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761533.403292:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761533.403297:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761533/real 1789761533] req@ffff93f850d41f80 x1876699565683456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761533.403304:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565683456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761533.403310:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761533.403312:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761533.403313:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565683456:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761533.466766:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761534.490805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761535.514763:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761536.538820:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761537.562793:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761538.522725:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761538.522733:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761538.522737:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761538.522747:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1473 00000100:00080000:0.0:1789761538.522756:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761538.522762:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761538.522795:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565683584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761538.522816:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761538.522819:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761538.522821:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761538.522838:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1473 00000100:00080000:0.0:1789761538.522842:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761538.522845:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761538.522855:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761538.522865:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565683712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761538.522875:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761538.522879:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:3.0:1789761538.522879:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565683584:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761538.522884:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761538.522897:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1473 00000100:00080000:0.0:1789761538.522901:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761538.522904:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761538.522907:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761538.522918:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565683840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761538.522926:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761538.522928:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761538.522930:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761538.522933:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1473 00000100:00080000:0.0:1789761538.522935:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761538.522938:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761538.522941:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761538.522950:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565683968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761538.522980:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761538.523049:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761538/real 1789761538] req@ffff93f850d43100 x1876699565683584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761538.523074:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565683584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761538.523089:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761538.523094:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761538.523099:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565683584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761538.523132:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565683712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761538.523155:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761538.523166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761538/real 1789761538] req@ffff93f850d41180 x1876699565683712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761538.523175:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565683712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761538.523181:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761538.523183:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761538.523186:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565683712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761538.523190:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565683840:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761538.523213:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761538.523221:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761538/real 1789761538] req@ffff93f850d40000 x1876699565683840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761538.523229:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565683840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761538.523234:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761538.523237:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761538.523239:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565683840:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761538.523243:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565683968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761538.523256:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761538.523262:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761538/real 1789761538] req@ffff93f850d42680 x1876699565683968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761538.523270:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565683968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761538.523275:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761538.523277:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761538.523279:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565683968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761538.586801:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761539.610810:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761540.634797:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761541.658766:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761542.682787:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761543.642803:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761543.642814:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761543.642818:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761543.642829:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1478 00000100:00080000:0.0:1789761543.642839:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761543.642846:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761543.642889:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565684096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761543.642917:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761543.642920:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761543.642922:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761543.642925:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1478 00000100:00080000:0.0:1789761543.642927:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761543.642930:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761543.642939:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761543.642949:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565684224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761543.642957:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761543.642960:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761543.642961:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761543.642967:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1478 00000100:00100000:3.0:1789761543.642967:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565684096:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761543.642972:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761543.642977:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761543.642980:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761543.642988:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565684352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761543.643000:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761543.643003:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761543.643004:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761543.643007:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1478 00000100:00080000:0.0:1789761543.643010:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761543.643013:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761543.643016:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761543.643023:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565684480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761543.643051:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761543.643110:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761543/real 1789761543] req@ffff93f850d41c00 x1876699565684096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761543.643137:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565684096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761543.643151:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761543.643156:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761543.643161:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565684096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761543.643189:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565684224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761543.643214:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761543.643227:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761543/real 1789761543] req@ffff93f850d40e00 x1876699565684224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761543.643240:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565684224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761543.643250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761543.643253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761543.643257:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565684224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761543.643263:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565684352:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761543.643285:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761543.643297:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761543/real 1789761543] req@ffff93f850d40700 x1876699565684352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761543.643310:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565684352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761543.643319:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761543.643322:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761543.643326:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565684352:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761543.643332:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565684480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761543.643349:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761543.643359:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761543/real 1789761543] req@ffff93f850d43b80 x1876699565684480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761543.643372:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565684480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761543.643381:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761543.643384:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761543.643387:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565684480:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761543.706844:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761544.730833:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761545.754840:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761546.778793:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761547.802802:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761548.762728:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761548.762736:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761548.762739:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761548.762759:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1483 00000100:00080000:1.0:1789761548.762767:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761548.762774:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761548.762807:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565684608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761548.762829:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761548.762832:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761548.762834:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761548.762837:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1483 00000100:00080000:1.0:1789761548.762839:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761548.762841:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761548.762849:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761548.762860:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565684736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761548.762869:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1789761548.762873:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565684608:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761548.762874:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761548.762877:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761548.762885:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1483 00000100:00080000:1.0:1789761548.762887:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761548.762890:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761548.762892:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761548.762901:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565684864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761548.762908:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761548.762910:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761548.762912:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761548.762914:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1483 00000100:00080000:1.0:1789761548.762917:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761548.762920:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761548.762923:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761548.762930:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565684992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761548.762943:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761548.762985:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761548/real 1789761548] req@ffff93f844ccb100 x1876699565684608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761548.763000:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565684608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761548.763009:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761548.763012:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761548.763016:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565684608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761548.763042:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565684736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761548.763067:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761548.763080:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761548/real 1789761548] req@ffff93f844cc9c00 x1876699565684736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761548.763095:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565684736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761548.763104:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761548.763108:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761548.763112:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565684736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761548.763118:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565684864:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761548.763137:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761548.763148:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761548/real 1789761548] req@ffff93f844cc8e00 x1876699565684864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761548.763156:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565684864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761548.763162:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761548.763164:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761548.763166:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565684864:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761548.763169:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565684992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761548.763182:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761548.763187:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761548/real 1789761548] req@ffff93f844cc9500 x1876699565684992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761548.763194:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565684992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761548.763204:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761548.763207:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761548.763209:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565684992:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761548.826769:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761549.850794:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761550.874827:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761551.898816:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761552.922837:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761553.882777:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761553.882789:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761553.882794:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761553.882805:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1488 00000100:00080000:0.0:1789761553.882816:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761553.882824:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761553.882865:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565685120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761553.882894:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761553.882899:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761553.882901:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761553.882905:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1488 00000100:00080000:0.0:1789761553.882909:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761553.882915:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761553.882931:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761553.882944:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565685120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761553.882945:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565685248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761553.882956:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761553.882960:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761553.882961:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761553.882965:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1488 00000100:00080000:0.0:1789761553.882968:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761553.882972:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761553.882977:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761553.882989:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565685376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761553.882999:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761553.883002:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761553.883004:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761553.883008:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1488 00000100:00080000:0.0:1789761553.883011:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761553.883014:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761553.883019:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789761553.883021:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761553.883029:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565685504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761553.883078:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761553/real 1789761553] req@ffff93f850d40380 x1876699565685120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761553.883105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565685120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761553.883114:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761553.883117:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761553.883122:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565685120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761553.883151:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565685248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761553.883179:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761553.883194:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761553/real 1789761553] req@ffff93f850d42300 x1876699565685248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761553.883206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565685248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761553.883213:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761553.883215:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761553.883217:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565685248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761553.883222:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565685376:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761553.883238:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761553.883247:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761553/real 1789761553] req@ffff93f850d43b80 x1876699565685376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761553.883254:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565685376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761553.883260:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761553.883262:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761553.883264:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565685376:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761553.883267:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565685504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761553.883278:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761553.883284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761553/real 1789761553] req@ffff93f850d40700 x1876699565685504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761553.883297:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565685504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761553.883305:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761553.883307:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761553.883310:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565685504:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761553.946830:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761554.970807:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761555.994794:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761557.018797:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761558.042802:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761559.002754:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761559.002765:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761559.002770:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761559.002781:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1493 00000100:00080000:2.0:1789761559.002792:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761559.002802:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761559.002845:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565685632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761559.002870:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761559.002874:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761559.002875:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761559.002878:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1493 00000100:00080000:2.0:1789761559.002881:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761559.002883:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761559.002892:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761559.002901:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565685760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761559.002909:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761559.002913:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761559.002914:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761559.002917:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1493 00000100:00080000:2.0:1789761559.002919:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761559.002924:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761559.002924:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565685632:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789761559.002932:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761559.002940:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565685888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761559.002948:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761559.002951:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761559.002951:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761559.002954:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1493 00000100:00080000:2.0:1789761559.002956:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761559.002960:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761559.002963:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761559.002971:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565686016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789761559.002996:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761559.003047:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761559/real 1789761559] req@ffff93f8517c1880 x1876699565685632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761559.003066:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565685632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761559.003076:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761559.003080:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761559.003083:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565685632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761559.003089:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565685760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761559.003105:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761559.003114:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761559/real 1789761559] req@ffff93f8517c3800 x1876699565685760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761559.003123:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565685760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761559.003129:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761559.003131:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761559.003133:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565685760:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761559.003158:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565685888:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761559.003171:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761559.003177:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761559/real 1789761559] req@ffff93f8517c3480 x1876699565685888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761559.003185:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565685888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761559.003190:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761559.003192:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761559.003194:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565685888:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761559.003198:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565686016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761559.003209:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761559.003215:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761559/real 1789761559] req@ffff93f8517c0e00 x1876699565686016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761559.003222:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565686016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761559.003227:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761559.003229:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761559.003231:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565686016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761559.066812:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761560.090855:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761561.114796:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761562.138795:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761563.162814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761564.122886:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761564.122908:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761564.122917:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761564.122934:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1499 00000100:00080000:0.0:1789761564.122949:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761564.122961:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761564.123017:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565686144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761564.123069:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761564.123074:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761564.123076:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761564.123081:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1499 00000100:00080000:0.0:1789761564.123085:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761564.123091:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761564.123104:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761564.123120:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565686144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761564.123124:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565686272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761564.123142:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761564.123146:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761564.123148:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761564.123153:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1499 00000100:00080000:0.0:1789761564.123158:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761564.123162:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761564.123169:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761564.123185:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565686400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761564.123203:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761564.123207:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761564.123209:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761564.123224:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1499 00000100:00080000:0.0:1789761564.123230:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1789761564.123236:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789761564.123240:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761564.123248:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761564.123264:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565686528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761564.123333:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761564/real 1789761564] req@ffff93f850d40e00 x1876699565686144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761564.123359:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565686144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761564.123373:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761564.123379:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761564.123384:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565686144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761564.123416:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565686272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761564.123450:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761564.123467:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761564/real 1789761564] req@ffff93f850d41c00 x1876699565686272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761564.123482:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565686272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761564.123493:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761564.123497:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761564.123502:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565686272:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761564.123511:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565686400:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761564.123537:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761564.123548:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761564/real 1789761564] req@ffff93f850d42680 x1876699565686400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761564.123557:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565686400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761564.123564:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761564.123567:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761564.123570:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565686400:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761564.123577:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565686528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761564.123593:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761564.123601:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761564/real 1789761564] req@ffff93f850d40000 x1876699565686528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761564.123760:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565686528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761564.123887:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761564.123892:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761564.123896:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565686528:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761564.186781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761565.210847:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761566.234773:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761567.258891:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761568.282778:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761569.242770:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761569.242781:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761569.242787:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761569.242799:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1504 00000100:00080000:3.0:1789761569.242812:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761569.242822:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761569.242875:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565686656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761569.242915:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761569.242919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761569.242922:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761569.242927:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1504 00000100:00080000:3.0:1789761569.242932:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761569.242936:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761569.242945:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761569.242951:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565686656:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761569.242960:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565686784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761569.242970:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761569.242973:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761569.242975:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761569.242978:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1504 00000100:00080000:3.0:1789761569.242980:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761569.242982:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761569.242987:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761569.242996:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565686912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761569.243004:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761569.243007:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761569.243008:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761569.243012:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1504 00000100:00080000:3.0:1789761569.243014:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761569.243016:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761569.243019:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789761569.243024:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761569.243029:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565687040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761569.243079:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761569/real 1789761569] req@ffff93f844d64700 x1876699565686656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761569.243097:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565686656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761569.243106:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761569.243110:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761569.243113:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565686656:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761569.243139:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565686784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761569.243161:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761569.243176:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761569/real 1789761569] req@ffff93f844d67b80 x1876699565686784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761569.243191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565686784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761569.243201:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761569.243205:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761569.243207:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565686784:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761569.243212:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565686912:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761569.243231:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761569.243239:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761569/real 1789761569] req@ffff93f844d66680 x1876699565686912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761569.243247:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565686912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761569.243254:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761569.243256:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761569.243258:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565686912:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761569.243262:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565687040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761569.243272:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761569.243278:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761569/real 1789761569] req@ffff93f844d65880 x1876699565687040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761569.243287:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565687040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761569.243292:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761569.243294:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761569.243296:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565687040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761569.306846:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761570.330819:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761571.354781:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761572.378776:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761573.402748:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761574.362793:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761574.362803:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761574.362807:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761574.362818:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1509 00000100:00080000:0.0:1789761574.362829:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761574.362836:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761574.362879:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565687168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761574.362904:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761574.362907:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761574.362909:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761574.362912:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1509 00000100:00080000:0.0:1789761574.362914:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761574.362917:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761574.362926:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761574.362935:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565687296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761574.362943:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761574.362946:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761574.362948:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761574.362951:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1509 00000100:00080000:0.0:1789761574.362953:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761574.362955:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761574.362958:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761574.362964:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565687424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761574.362974:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761574.362979:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761574.362981:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789761574.362984:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565687168:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761574.362987:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1509 00000100:00080000:0.0:1789761574.362994:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761574.363002:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761574.363006:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761574.363017:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565687552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761574.363068:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761574.363111:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761574/real 1789761574] req@ffff93f850d41180 x1876699565687168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761574.363129:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565687168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761574.363140:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761574.363145:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761574.363149:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565687168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761574.363158:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565687296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761574.363181:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761574.363195:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761574/real 1789761574] req@ffff93f850d43100 x1876699565687296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761574.363206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565687296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761574.363221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761574.363225:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761574.363229:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565687296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761574.363254:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565687424:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761574.363274:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761574.363285:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761574/real 1789761574] req@ffff93f850d41f80 x1876699565687424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761574.363296:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565687424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761574.363304:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761574.363308:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761574.363311:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565687424:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761574.363317:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565687552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761574.363335:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761574.363346:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761574/real 1789761574] req@ffff93f850d41500 x1876699565687552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761574.363357:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565687552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761574.363365:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761574.363368:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761574.363372:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565687552:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761574.426784:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761575.450783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761576.474783:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761577.498789:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761578.522867:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761579.482750:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761579.482758:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761579.482761:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761579.482769:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1514 00000100:00080000:3.0:1789761579.482777:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761579.482783:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761579.482818:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565687680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761579.482841:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761579.482844:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761579.482845:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761579.482848:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1514 00000100:00080000:3.0:1789761579.482850:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761579.482852:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761579.482861:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761579.482870:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565687808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761579.482879:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761579.482882:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761579.482883:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761579.482886:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1514 00000100:00080000:3.0:1789761579.482889:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761579.482896:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761579.482899:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565687680:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789761579.482902:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761579.482919:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565687936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761579.482928:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761579.482931:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761579.482932:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761579.482936:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1514 00000100:00080000:3.0:1789761579.482939:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761579.482941:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761579.482944:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761579.482951:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565688064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761579.482990:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761579.483043:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761579/real 1789761579] req@ffff93f844d67480 x1876699565687680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761579.483061:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565687680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761579.483070:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761579.483073:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761579.483076:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565687680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761579.483102:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565687808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761579.483119:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761579.483129:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761579/real 1789761579] req@ffff93f844d67800 x1876699565687808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761579.483137:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565687808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761579.483143:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761579.483146:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761579.483148:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565687808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761579.483153:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565687936:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761579.483165:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761579.483172:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761579/real 1789761579] req@ffff93f844d67100 x1876699565687936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761579.483180:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565687936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761579.483185:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761579.483187:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761579.483189:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565687936:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761579.483193:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565688064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761579.483204:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761579.483211:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761579/real 1789761579] req@ffff93f844d66a00 x1876699565688064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761579.483219:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565688064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761579.483224:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761579.483225:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761579.483227:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565688064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761579.546783:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761580.570809:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761581.594786:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761582.618845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761583.642781:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761584.602741:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761584.602754:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761584.602760:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761584.602774:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1519 00000100:00080000:3.0:1789761584.602786:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761584.602795:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761584.602843:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565688192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761584.602879:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761584.602883:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761584.602885:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761584.602890:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1519 00000100:00080000:3.0:1789761584.602894:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761584.602899:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761584.602912:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761584.602927:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565688320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761584.602931:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565688192:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761584.602948:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761584.602952:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761584.602954:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761584.602959:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1519 00000100:00080000:3.0:1789761584.602962:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761584.602966:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761584.602971:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761584.602984:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565688448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761584.602994:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761584.602998:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761584.603000:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761584.603003:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1519 00000100:00080000:3.0:1789761584.603007:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761584.603013:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761584.603018:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761584.603030:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565688576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761584.603084:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565688320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761584.603125:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565688448:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761584.603158:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565688576:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761584.666835:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761585.690800:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761586.714839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789761587.674907:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789761587.674950:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789761587.674966:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761587.675008:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761587.675019:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761587.675045:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789761587.675054:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761584/real 1789761587] req@ffff93f844d64000 x1876699565688320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761587.675081:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565688320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761587.675097:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761587.675102:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761587.675106:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565688320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761587.675114:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761584/real 1789761587] req@ffff93f844d66d80 x1876699565688448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761587.675126:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565688448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761587.675140:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761587.675144:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761587.675147:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565688448:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761587.675154:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761584/real 1789761587] req@ffff93f844d64e00 x1876699565688576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761587.675171:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565688576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761587.675179:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761587.675182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761587.675185:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565688576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761587.675193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761584/real 1789761587] req@ffff93f844d65c00 x1876699565688192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761587.675204:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565688192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761587.675211:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761587.675215:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761587.675219:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565688192:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761587.738820:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761588.762826:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761589.722723:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761589.722737:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761589.722742:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761589.722754:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1524 00000100:00080000:3.0:1789761589.722769:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761589.722778:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761589.722832:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565688704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761589.722862:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761589.722866:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761589.722868:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761589.722873:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1524 00000100:00080000:3.0:1789761589.722878:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761589.722883:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761589.722893:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761589.722897:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565688704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761589.722907:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565688832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761589.722918:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761589.722921:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761589.722923:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761589.722927:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1524 00000100:00080000:3.0:1789761589.722930:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761589.722934:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761589.722939:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761589.722948:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565688960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761589.722958:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761589.722962:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761589.722963:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761589.722966:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1524 00000800:00000100:1.0:1789761589.722970:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761589.722970:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761589.722974:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761589.722979:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761589.722989:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565689088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761589.723044:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761589/real 1789761589] req@ffff93f844d65500 x1876699565688704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761589.723067:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565688704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761589.723080:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761589.723085:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761589.723089:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565688704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761589.723122:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565688832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761589.723149:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761589.723166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761589/real 1789761589] req@ffff93f844d65f80 x1876699565688832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761589.723180:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565688832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761589.723190:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761589.723193:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761589.723197:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565688832:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761589.723203:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565688960:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761589.723226:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761589.723239:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761589/real 1789761589] req@ffff93f844d64a80 x1876699565688960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761589.723253:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565688960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761589.723263:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761589.723267:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761589.723271:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565688960:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761589.723278:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565689088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761589.723300:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761589.723314:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761589/real 1789761589] req@ffff93f844d64380 x1876699565689088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761589.723327:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565689088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761589.723335:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761589.723338:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761589.723343:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565689088:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761589.786774:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761590.810847:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761591.834839:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761592.858794:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761593.882785:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761594.842729:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761594.842742:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761594.842748:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761594.842760:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1529 00000100:00080000:3.0:1789761594.842772:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761594.842780:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761594.842826:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565689216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761594.842853:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761594.842856:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761594.842857:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761594.842861:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1529 00000100:00080000:3.0:1789761594.842864:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761594.842866:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761594.842875:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761594.842886:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565689344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761594.842894:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761594.842897:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761594.842898:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789761594.842906:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565689216:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761594.842909:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1529 00000100:00080000:3.0:1789761594.842914:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761594.842918:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761594.842921:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761594.842935:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565689472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761594.842946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761594.842950:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761594.842952:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761594.842956:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1529 00000100:00080000:3.0:1789761594.842962:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761594.842967:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761594.842973:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789761594.842989:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761594.842998:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565689600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761594.843047:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761594/real 1789761594] req@ffff93f844d66300 x1876699565689216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761594.843072:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565689216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761594.843086:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761594.843091:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761594.843094:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565689216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761594.843119:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565689344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761594.843140:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761594.843150:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761594/real 1789761594] req@ffff93f844d65180 x1876699565689344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761594.843159:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565689344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761594.843165:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761594.843168:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761594.843172:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565689344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761594.843179:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565689472:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761594.843201:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761594.843215:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761594/real 1789761594] req@ffff93f844d64380 x1876699565689472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761594.843229:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565689472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761594.843238:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761594.843242:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761594.843246:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565689472:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761594.843252:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565689600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761594.843274:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761594.843283:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761594/real 1789761594] req@ffff93f844d64a80 x1876699565689600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761594.843292:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565689600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761594.843298:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761594.843301:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761594.843303:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565689600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761594.906822:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761595.930801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761596.954799:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761597.978820:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761599.002832:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761599.962727:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761599.962741:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761599.962748:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761599.962766:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1534 00000100:00080000:3.0:1789761599.962781:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761599.962797:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761599.962845:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565689728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761599.962873:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761599.962877:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761599.962880:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761599.962886:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1534 00000100:00080000:3.0:1789761599.962892:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761599.962895:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761599.962907:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761599.962916:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565689728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761599.962931:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565689856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761599.962946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761599.962950:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761599.962952:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761599.962957:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1534 00000100:00080000:3.0:1789761599.962960:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761599.962964:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1789761599.962995:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1789761599.963009:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761599.963024:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565689984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761599.963036:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761599.963039:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761599.963041:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789761599.963047:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761599/real 1789761599] req@ffff93f844d65f80 x1876699565689728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761599.963058:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1534 00000100:00100000:1.0:1789761599.963069:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565689728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761599.963079:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761599.963082:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761599.963086:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565689728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761599.963111:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565689856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761599.963131:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761599.963142:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761599/real 1789761599] req@ffff93f844d65500 x1876699565689856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761599.963152:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565689856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761599.963159:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761599.963163:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761599.963174:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565689856:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1789761599.963174:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761599.963187:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761599.963192:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565689984:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761599.963212:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761599.963233:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761599/real 1789761599] req@ffff93f844d65c00 x1876699565689984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1789761599.963235:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761599.963266:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565690112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761599.963274:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565689984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761599.963295:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761599.963301:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761599.963304:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565689984:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761599.963329:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565690112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761599.963343:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761599.963350:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761599/real 1789761599] req@ffff93f844d64e00 x1876699565690112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761599.963357:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565690112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761599.963362:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761599.963382:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761599.963385:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565690112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761600.026831:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761601.050799:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761602.074785:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761603.098781:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761604.122822:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761605.082790:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761605.082800:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761605.082805:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761605.082818:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1540 00000100:00080000:3.0:1789761605.082831:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761605.082841:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761605.082891:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565690240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761605.082924:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761605.082933:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761605.082935:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761605.082940:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1540 00000100:00080000:3.0:1789761605.082944:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761605.082950:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761605.082962:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761605.082977:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565690240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761605.082977:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565690368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761605.082998:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761605.083001:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761605.083002:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761605.083006:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1540 00000100:00080000:3.0:1789761605.083008:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761605.083011:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761605.083015:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761605.083025:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565690496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761605.083032:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761605.083035:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761605.083037:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761605.083040:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1540 00000100:00080000:3.0:1789761605.083042:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761605.083044:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761605.083047:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789761605.083052:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761605.083056:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565690624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761605.083105:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761605/real 1789761605] req@ffff93f844d66d80 x1876699565690240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761605.083125:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565690240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761605.083137:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761605.083140:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761605.083144:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565690240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761605.083171:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565690368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761605.083188:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761605.083197:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761605/real 1789761605] req@ffff93f844d64000 x1876699565690368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761605.083206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565690368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761605.083212:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761605.083215:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761605.083217:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565690368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761605.083220:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565690496:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761605.083232:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761605.083238:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761605/real 1789761605] req@ffff93f844d66a00 x1876699565690496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761605.083246:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565690496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761605.083251:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761605.083253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761605.083255:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565690496:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761605.083258:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565690624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761605.083280:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761605.083286:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761605/real 1789761605] req@ffff93f844d67100 x1876699565690624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761605.083293:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565690624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761605.083303:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761605.083306:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761605.083308:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565690624:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761605.146795:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761606.170850:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761607.194808:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761608.218862:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761609.242766:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761610.202765:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761610.202773:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761610.202777:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761610.202785:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1545 00000100:00080000:2.0:1789761610.202793:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761610.202799:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761610.202829:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565690752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761610.202858:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761610.202862:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761610.202863:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761610.202868:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1545 00000100:00080000:2.0:1789761610.202872:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761610.202877:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761610.202888:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761610.202903:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565690752:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761610.202905:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565690880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761610.202925:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761610.202929:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761610.202931:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761610.202935:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1545 00000100:00080000:2.0:1789761610.202940:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761610.202944:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761610.202950:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761610.202963:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565691008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761610.202976:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761610.202980:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761610.202983:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1789761610.202986:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761610.202990:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1545 00000100:00080000:2.0:1789761610.202995:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761610.202999:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761610.203003:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761610.203019:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565691136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789761610.203038:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761610/real 1789761610] req@ffff93f8517c2d80 x1876699565690752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761610.203057:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565690752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761610.203066:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761610.203069:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761610.203072:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565690752:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761610.203091:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565690880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761610.203109:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761610.203118:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761610/real 1789761610] req@ffff93f8517c2a00 x1876699565690880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761610.203126:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565690880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761610.203132:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761610.203134:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761610.203136:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565690880:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761610.203142:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565691008:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789761610.203162:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761610.203174:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761610/real 1789761610] req@ffff93f8517c3100 x1876699565691008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761610.203194:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565691008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761610.203204:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761610.203207:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761610.203211:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565691008:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761610.203218:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565691136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789761610.203239:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761610.203250:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761610/real 1789761610] req@ffff93f8517c1180 x1876699565691136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761610.203262:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565691136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761610.203271:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761610.203274:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761610.203277:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565691136:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761610.266806:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761611.290858:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761612.314850:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761613.338838:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761614.362818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761615.322757:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761615.322765:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761615.322769:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761615.322778:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1550 00000100:00080000:3.0:1789761615.322787:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761615.322793:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761615.322824:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565691264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761615.322848:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761615.322851:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761615.322852:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761615.322855:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1550 00000100:00080000:3.0:1789761615.322857:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761615.322860:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761615.322867:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761615.322875:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565691392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761615.322889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761615.322892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761615.322894:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789761615.322896:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565691264:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761615.322898:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1550 00000100:00080000:3.0:1789761615.322904:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761615.322908:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761615.322911:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761615.322919:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565691520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761615.322930:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761615.322933:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761615.322934:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761615.322936:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1550 00000100:00080000:3.0:1789761615.322939:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761615.322942:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761615.322945:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761615.322951:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565691648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761615.322965:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761615.323015:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761615/real 1789761615] req@ffff93f844d67800 x1876699565691264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761615.323036:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565691264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761615.323048:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761615.323052:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761615.323056:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565691264:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761615.323080:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565691392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761615.323101:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761615.323113:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761615/real 1789761615] req@ffff93f844d67480 x1876699565691392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761615.323126:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565691392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761615.323135:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761615.323138:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761615.323142:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565691392:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761615.323149:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565691520:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761615.323166:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761615.323176:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761615/real 1789761615] req@ffff93f844d65880 x1876699565691520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761615.323196:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565691520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761615.323206:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761615.323209:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761615.323212:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565691520:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761615.323218:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565691648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761615.323236:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761615.323246:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761615/real 1789761615] req@ffff93f844d66680 x1876699565691648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761615.323273:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565691648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761615.323281:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761615.323284:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761615.323288:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565691648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761615.386753:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761616.410773:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761617.434798:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761618.458794:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761619.482754:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761620.442753:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761620.442764:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761620.442769:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761620.442780:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1555 00000100:00080000:3.0:1789761620.442791:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761620.442799:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761620.442851:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565691776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761620.442888:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761620.442891:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761620.442893:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761620.442897:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1555 00000100:00080000:3.0:1789761620.442900:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761620.442903:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761620.442913:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761620.442924:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565691904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761620.442933:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789761620.442938:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565691776:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761620.442938:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761620.442942:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761620.442949:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1555 00000100:00080000:3.0:1789761620.442952:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761620.442954:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761620.442958:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761620.442968:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565692032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761620.442976:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761620.442979:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761620.442980:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761620.442983:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1555 00000100:00080000:3.0:1789761620.442985:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761620.442988:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761620.442991:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761620.442999:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565692160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761620.443021:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761620.443083:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761620/real 1789761620] req@ffff93f844d67b80 x1876699565691776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761620.443106:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565691776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761620.443119:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761620.443124:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761620.443129:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565691776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761620.443160:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565691904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761620.443187:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761620.443200:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761620/real 1789761620] req@ffff93f844d64700 x1876699565691904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761620.443226:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565691904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761620.443234:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761620.443238:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761620.443241:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565691904:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761620.443248:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565692032:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761620.443272:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761620.443285:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761620/real 1789761620] req@ffff93f844d66680 x1876699565692032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761620.443296:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565692032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761620.443306:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761620.443309:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761620.443312:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565692032:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761620.443316:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565692160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761620.443330:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761620.443338:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761620/real 1789761620] req@ffff93f844d65880 x1876699565692160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761620.443345:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565692160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761620.443351:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761620.443353:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761620.443355:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565692160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761620.506793:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761621.530827:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761622.554774:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761623.578853:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761624.602819:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761625.562790:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761625.562805:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761625.562809:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761625.562822:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1560 00000100:00080000:2.0:1789761625.562837:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761625.562847:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761625.562896:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565692288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761625.562924:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761625.562927:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761625.562929:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761625.562932:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1560 00000100:00080000:2.0:1789761625.562935:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761625.562938:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761625.562948:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761625.562959:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565692416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761625.562969:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789761625.562970:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565692288:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761625.562978:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761625.562981:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761625.562988:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1560 00000100:00080000:2.0:1789761625.562993:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761625.562997:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761625.563004:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761625.563020:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565692544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761625.563032:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761625.563036:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761625.563039:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789761625.563041:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761625.563048:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1560 00000100:00080000:2.0:1789761625.563055:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761625.563061:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761625.563068:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761625.563083:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565692672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761625.563104:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761625/real 1789761625] req@ffff93f8517c0a80 x1876699565692288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761625.563122:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565692288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761625.563137:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761625.563143:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761625.563148:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565692288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761625.563190:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565692416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761625.563211:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761625.563222:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761625/real 1789761625] req@ffff93f8517c1500 x1876699565692416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761625.563231:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565692416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761625.563238:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761625.563241:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761625.563245:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565692416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761625.563251:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565692544:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761625.563274:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761625.563286:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761625/real 1789761625] req@ffff93f8517c0380 x1876699565692544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761625.563298:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565692544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761625.563304:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761625.563306:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761625.563308:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565692544:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761625.563312:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565692672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761625.563327:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761625.563335:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761625/real 1789761625] req@ffff93f8517c2680 x1876699565692672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761625.563343:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565692672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761625.563348:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761625.563350:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761625.563352:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565692672:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761625.626955:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761626.650816:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761627.674818:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761628.698768:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761629.722814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761630.682907:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761630.682921:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761630.682929:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761630.682944:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1565 00000100:00080000:3.0:1789761630.682959:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761630.682968:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761630.683019:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565692800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761630.683050:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761630.683054:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761630.683057:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761630.683061:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1565 00000100:00080000:3.0:1789761630.683065:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761630.683069:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761630.683089:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565692800:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789761630.683092:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761630.683123:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565692928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761630.683151:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761630.683156:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761630.683158:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761630.683163:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1565 00000100:00080000:3.0:1789761630.683169:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761630.683174:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761630.683182:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789761630.683191:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761630.683202:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565693056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761630.683218:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761630.683222:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761630.683225:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761630.683231:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1565 00000100:00080000:3.0:1789761630.683236:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789761630.683241:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761630/real 1789761630] req@ffff93f844d67480 x1876699565692800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761630.683246:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761630.683258:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761630.683271:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565692800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761630.683277:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565693184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761630.683310:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761630.683318:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761630.683327:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565692800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761630.683356:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565692928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761630.683382:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761630.683395:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761630/real 1789761630] req@ffff93f844d67800 x1876699565692928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761630.683408:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565692928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761630.683424:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761630.683428:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761630.683430:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565692928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761630.683435:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565693056:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761630.683452:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761630.683459:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761630/real 1789761630] req@ffff93f844d67100 x1876699565693056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761630.683467:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565693056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761630.683473:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761630.683475:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761630.683477:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565693056:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761630.683481:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565693184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761630.683492:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761630.683500:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761630/real 1789761630] req@ffff93f844d66a00 x1876699565693184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761630.683512:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565693184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761630.683519:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761630.683522:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761630.683526:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565693184:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761630.746793:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761631.770777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761632.794792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761633.818805:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761634.842773:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761635.802726:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761635.802734:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761635.802738:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761635.802746:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1570 00000100:00080000:0.0:1789761635.802754:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761635.802761:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761635.802791:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565693312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761635.802817:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761635.802820:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761635.802821:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761635.802824:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1570 00000100:00080000:0.0:1789761635.802827:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761635.802829:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761635.802838:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761635.802850:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565693440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761635.802858:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789761635.802859:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565693312:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761635.802864:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761635.802868:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761635.802872:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1570 00000100:00080000:0.0:1789761635.802874:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761635.802878:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761635.802882:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761635.802892:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565693568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761635.802899:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761635.802902:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761635.802904:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761635.802907:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1570 00000100:00080000:0.0:1789761635.802916:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761635.802920:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1789761635.802924:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1789761635.802927:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761635.802938:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565693696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761635.802963:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761635/real 1789761635] req@ffff93f850d43480 x1876699565693312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761635.802977:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565693312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761635.802991:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761635.802995:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761635.802998:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565693312:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761635.803021:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565693440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761635.803037:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761635.803046:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761635/real 1789761635] req@ffff93f850d43800 x1876699565693440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761635.803058:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565693440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761635.803066:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761635.803070:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761635.803073:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565693440:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761635.803079:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565693568:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761635.803115:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761635.803127:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761635/real 1789761635] req@ffff93f850d41500 x1876699565693568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761635.803140:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565693568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761635.803149:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761635.803152:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761635.803155:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565693568:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761635.803161:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565693696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761635.803185:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761635.803212:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761635/real 1789761635] req@ffff93f850d41f80 x1876699565693696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761635.803226:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565693696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761635.803234:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761635.803239:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761635.803244:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565693696:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761635.866801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761636.890791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761637.914781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761638.938839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761639.962784:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761640.922757:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761640.922767:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761640.922771:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761640.922782:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1575 00000100:00080000:3.0:1789761640.922792:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761640.922800:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761640.922840:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565693824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761640.922876:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761640.922881:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761640.922883:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761640.922887:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1575 00000100:00080000:3.0:1789761640.922893:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761640.922898:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761640.922918:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565693824:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789761640.922920:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761640.922939:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565693952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761640.922952:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761640.922956:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761640.922958:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761640.922962:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1575 00000100:00080000:3.0:1789761640.922966:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761640.922971:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761640.922986:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789761640.922988:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761640.923006:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565694080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761640.923018:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761640.923022:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761640.923023:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761640.923028:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1575 00000100:00080000:3.0:1789761640.923033:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789761640.923038:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761640/real 1789761640] req@ffff93f844d64000 x1876699565693824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761640.923040:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761640.923049:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761640.923070:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565693824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761640.923079:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565694208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761640.923082:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761640.923090:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761640.923097:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565693824:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761640.923135:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565693952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761640.923160:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761640.923174:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761640/real 1789761640] req@ffff93f844d66d80 x1876699565693952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761640.923184:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565693952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761640.923190:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761640.923193:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761640.923195:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565693952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761640.923199:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565694080:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761640.923213:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761640.923220:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761640/real 1789761640] req@ffff93f844d64e00 x1876699565694080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761640.923228:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565694080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761640.923234:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761640.923236:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761640.923238:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565694080:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761640.923241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565694208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761640.923252:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761640.923258:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761640/real 1789761640] req@ffff93f844d65c00 x1876699565694208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761640.923265:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565694208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761640.923275:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761640.923277:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761640.923279:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565694208:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761640.986799:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761642.010803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761643.034813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761644.058789:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761645.082832:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761646.042739:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761646.042750:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761646.042755:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761646.042766:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1581 00000100:00080000:3.0:1789761646.042776:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761646.042784:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761646.042825:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565694336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761646.042853:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761646.042856:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761646.042858:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761646.042861:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1581 00000100:00080000:3.0:1789761646.042863:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761646.042867:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761646.042876:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761646.042886:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565694336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761646.042888:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565694464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761646.042901:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761646.042903:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761646.042905:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761646.042908:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1581 00000100:00080000:3.0:1789761646.042910:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761646.042913:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761646.042916:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761646.042924:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565694592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761646.042936:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761646.042939:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761646.042941:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761646.042944:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1581 00000100:00080000:3.0:1789761646.042953:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761646.042956:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761646.042959:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761646.042966:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565694720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761646.042973:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761646.043038:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761646/real 1789761646] req@ffff93f844d65500 x1876699565694336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761646.043060:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565694336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761646.043072:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761646.043078:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761646.043082:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565694336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761646.043114:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565694464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761646.043142:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761646.043157:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761646/real 1789761646] req@ffff93f844d65f80 x1876699565694464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761646.043172:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565694464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761646.043182:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761646.043185:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761646.043187:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565694464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761646.043193:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565694592:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761646.043210:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761646.043219:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761646/real 1789761646] req@ffff93f844d64a80 x1876699565694592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761646.043227:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565694592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761646.043233:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761646.043236:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761646.043238:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565694592:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761646.043241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565694720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761646.043252:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761646.043258:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761646/real 1789761646] req@ffff93f844d64380 x1876699565694720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761646.043265:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565694720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761646.043271:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761646.043273:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761646.043274:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565694720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761646.106784:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761647.130812:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761648.154775:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761649.178874:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761650.202882:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761651.162742:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761651.162754:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761651.162759:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761651.162771:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1586 00000100:00080000:1.0:1789761651.162781:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761651.162788:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761651.162841:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565694848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761651.162863:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761651.162866:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761651.162867:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761651.162870:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1586 00000100:00080000:1.0:1789761651.162873:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761651.162876:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761651.162884:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761651.162895:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565694976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761651.162901:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565694848:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761651.162906:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761651.162910:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761651.162912:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761651.162915:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1586 00000100:00080000:1.0:1789761651.162917:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761651.162919:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761651.162922:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761651.162934:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565695104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761651.162951:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761651.162955:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761651.162957:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761651.162962:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1586 00000100:00080000:1.0:1789761651.162968:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761651.162979:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761651.162998:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761651.163012:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565695232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761651.163037:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565694976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761651.163076:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565695104:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761651.163094:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565695232:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761651.226804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761652.250776:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761653.274798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789761654.234952:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789761654.235016:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789761654.235037:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761654.235085:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761654.235098:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761654.235112:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789761654.235144:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761651/real 1789761654] req@ffff93f844ccad80 x1876699565694976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761654.235178:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565694976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761654.235190:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761654.235194:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761654.235199:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565694976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761654.235207:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761651/real 1789761654] req@ffff93f844cc9500 x1876699565695104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761654.235220:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565695104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761654.235228:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761654.235232:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761654.235235:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565695104:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761654.235243:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761651/real 1789761654] req@ffff93f844cc8e00 x1876699565695232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761654.235256:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565695232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761654.235271:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761654.235274:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761654.235278:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565695232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761654.235286:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761651/real 1789761654] req@ffff93f844cca680 x1876699565694848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761654.235300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565694848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761654.235308:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761654.235312:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761654.235315:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565694848:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761654.298799:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761655.322814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761656.282779:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761656.282794:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761656.282800:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761656.282813:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1591 00000100:00080000:3.0:1789761656.282827:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761656.282837:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761656.282886:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565695360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761656.282921:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761656.282930:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761656.282933:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761656.282938:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1591 00000100:00080000:3.0:1789761656.282953:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761656.282960:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761656.282973:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761656.282992:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565695488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761656.282998:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565695360:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761656.283010:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761656.283016:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761656.283018:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761656.283030:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1591 00000100:00080000:3.0:1789761656.283035:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761656.283040:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761656.283047:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761656.283066:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565695616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761656.283081:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1789761656.283084:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761656.283087:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761656.283092:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761656.283100:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1591 00000100:00080000:3.0:1789761656.283106:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761656.283112:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761656.283119:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761656.283140:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565695744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761656.283153:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761656/real 1789761656] req@ffff93f844d65180 x1876699565695360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761656.283180:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565695360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761656.283194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761656.283200:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761656.283204:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565695360:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761656.283233:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565695488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761656.283264:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761656.283278:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761656/real 1789761656] req@ffff93f844d66300 x1876699565695488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761656.283290:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565695488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761656.283299:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761656.283303:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761656.283306:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565695488:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761656.283315:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565695616:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761656.283337:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761656.283349:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761656/real 1789761656] req@ffff93f844d64380 x1876699565695616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761656.283361:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565695616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761656.283369:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761656.283373:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761656.283376:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565695616:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761656.283383:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565695744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761656.283406:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761656.283418:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761656/real 1789761656] req@ffff93f844d64a80 x1876699565695744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761656.283442:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565695744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761656.283450:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761656.283453:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761656.283457:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565695744:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761656.346817:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761657.370787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761658.394770:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761659.418749:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761660.442841:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761661.402812:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761661.402821:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761661.402824:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761661.402834:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1596 00000100:00080000:1.0:1789761661.402843:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761661.402848:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761661.402880:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565695872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761661.402914:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761661.402918:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761661.402919:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761661.402922:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1596 00000100:00080000:1.0:1789761661.402925:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761661.402928:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761661.402938:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761661.402953:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565696000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761661.402965:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789761661.402975:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565695872:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761661.402983:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761661.402993:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761661.402999:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1596 00000100:00080000:1.0:1789761661.403002:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761661.403007:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761661.403011:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761661.403023:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565696128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761661.403032:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761661.403035:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761661.403037:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761661.403041:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1596 00000100:00080000:1.0:1789761661.403043:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761661.403045:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761661.403048:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761661.403055:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565696256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761661.403072:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761661.403163:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761661/real 1789761661] req@ffff93f844cc9c00 x1876699565695872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761661.403184:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565695872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761661.403194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761661.403198:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761661.403202:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565695872:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761661.403227:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565696000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761661.403249:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761661.403260:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761661/real 1789761661] req@ffff93f844ccb100 x1876699565696000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761661.403269:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565696000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761661.403280:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761661.403282:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761661.403284:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565696000:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761661.403289:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565696128:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761661.403303:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761661.403309:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761661/real 1789761661] req@ffff93f844ccb480 x1876699565696128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761661.403317:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565696128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761661.403323:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761661.403325:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761661.403327:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565696128:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761661.403332:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565696256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761661.403343:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761661.403350:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761661/real 1789761661] req@ffff93f844ccb800 x1876699565696256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761661.403358:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565696256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761661.403363:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761661.403364:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761661.403368:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565696256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761661.466882:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761662.490826:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761663.514749:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761664.538791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761665.562782:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761666.522746:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761666.522754:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761666.522757:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761666.522766:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1601 00000100:00080000:0.0:1789761666.522773:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761666.522787:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761666.522826:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565696384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761666.522853:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761666.522857:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761666.522860:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761666.522877:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1601 00000100:00080000:0.0:1789761666.522883:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761666.522887:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761666.522894:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565696384:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1789761666.522896:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761666.522920:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565696512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761666.522933:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761666.522937:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761666.522949:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761666.522954:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1601 00000100:00080000:0.0:1789761666.522959:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789761666.522962:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789761666.522968:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761666.522974:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761666.522985:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565696640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761666.522995:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761666.523000:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:2.0:1789761666.523000:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761666/real 1789761666] req@ffff93f850d43100 x1876699565696384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761666.523017:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761666.523028:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1601 00000100:00100000:2.0:1789761666.523033:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565696384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761666.523035:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761666.523044:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:2.0:1789761666.523052:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:0.0:1789761666.523053:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00080000:2.0:1789761666.523063:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761666.523074:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565696768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761666.523074:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565696384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761666.523107:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565696512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761666.523133:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761666.523144:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761666/real 1789761666] req@ffff93f850d41180 x1876699565696512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761666.523153:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565696512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761666.523158:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761666.523161:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761666.523163:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565696512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761666.523167:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565696640:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761666.523180:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761666.523186:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761666/real 1789761666] req@ffff93f850d40000 x1876699565696640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761666.523194:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565696640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761666.523199:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761666.523201:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761666.523203:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565696640:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761666.523206:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565696768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761666.523216:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761666.523222:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761666/real 1789761666] req@ffff93f850d42680 x1876699565696768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761666.523230:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565696768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761666.523234:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761666.523236:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761666.523238:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565696768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761666.586771:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761667.610779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761668.634805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761669.658757:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761670.682788:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761671.642791:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761671.642802:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761671.642807:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761671.642819:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1606 00000100:00080000:3.0:1789761671.642843:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761671.642852:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761671.642895:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565696896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761671.642922:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761671.642925:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761671.642927:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761671.642931:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1606 00000100:00080000:3.0:1789761671.642935:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761671.642938:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761671.642948:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761671.642965:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565697024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761671.642975:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761671.642977:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761671.642980:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761671.642986:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1606 00000100:00080000:3.0:1789761671.642990:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761671.642994:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761671.642998:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761671.643009:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565697152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761671.643019:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761671.643023:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761671.643025:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761671.643036:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1606 00000100:00100000:2.0:1789761671.643039:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565696896:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761671.643040:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761671.643051:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761671.643061:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761671.643115:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761671.643154:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761671/real 1789761671] req@ffff93f844d65f80 x1876699565696896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761671.643172:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565697280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761671.643176:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565696896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761671.643221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761671.643228:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761671.643234:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565696896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761671.643241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565697024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761671.643262:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761671.643272:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761671/real 1789761671] req@ffff93f844d65500 x1876699565697024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761671.643280:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565697024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761671.643286:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761671.643288:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761671.643290:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565697024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761671.643313:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565697152:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761671.643325:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761671.643332:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761671/real 1789761671] req@ffff93f844d65c00 x1876699565697152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761671.643339:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565697152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761671.643345:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761671.643346:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761671.643348:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565697152:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761671.643353:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565697280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761671.643365:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761671.643372:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761671/real 1789761671] req@ffff93f844d64e00 x1876699565697280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761671.643379:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565697280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761671.643384:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761671.643386:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761671.643387:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565697280:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761671.706798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761672.730773:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761673.754793:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761674.778786:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761675.802753:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761676.762746:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761676.762757:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761676.762761:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761676.762772:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1611 00000100:00080000:3.0:1789761676.762803:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761676.762812:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761676.762859:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565697408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761676.762889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761676.762892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761676.762893:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761676.762897:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1611 00000100:00080000:3.0:1789761676.762900:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761676.762903:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761676.762913:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761676.762930:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565697536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761676.762942:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565697408:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761676.762946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761676.762952:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761676.762954:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761676.762959:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1611 00000100:00080000:3.0:1789761676.762964:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761676.762969:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761676.762976:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761676.762991:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565697664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761676.763002:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761676.763005:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761676.763007:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761676.763011:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1611 00000800:00000100:2.0:1789761676.763015:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761676.763020:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761676.763027:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761676.763034:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761676.763049:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565697792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761676.763070:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761676/real 1789761676] req@ffff93f844d66d80 x1876699565697408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761676.763099:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565697408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761676.763112:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761676.763117:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761676.763122:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565697408:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761676.763157:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565697536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761676.763177:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761676.763187:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761676/real 1789761676] req@ffff93f844d64000 x1876699565697536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761676.763197:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565697536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761676.763204:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761676.763206:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761676.763208:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565697536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761676.763212:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565697664:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761676.763225:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761676.763231:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761676/real 1789761676] req@ffff93f844d66a00 x1876699565697664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761676.763240:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565697664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761676.763245:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761676.763247:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761676.763249:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565697664:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761676.763252:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565697792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761676.763264:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761676.763269:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761676/real 1789761676] req@ffff93f844d67100 x1876699565697792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761676.763277:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565697792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761676.763287:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761676.763289:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761676.763292:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565697792:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761676.826849:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761677.850816:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761678.874811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761679.898802:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761680.922866:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761681.882742:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761681.882767:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761681.882773:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761681.882788:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1616 00000100:00080000:3.0:1789761681.882801:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761681.882810:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761681.882848:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565697920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761681.882877:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761681.882881:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761681.882883:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761681.882887:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1616 00000100:00080000:3.0:1789761681.882893:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761681.882898:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761681.882908:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761681.882922:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565698048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761681.882933:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565697920:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761681.882959:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761681.882964:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761681.882967:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761681.882973:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1616 00000100:00080000:3.0:1789761681.882977:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761681.882982:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761681.882989:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761681.883004:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565698176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761681.883017:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761681.883021:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761681.883024:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761681.883035:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1616 00000800:00000100:2.0:1789761681.883037:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761681.883040:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761681.883047:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761681.883052:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761681.883067:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565698304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761681.883117:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761681/real 1789761681] req@ffff93f844d67800 x1876699565697920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761681.883145:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565697920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761681.883159:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761681.883165:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761681.883169:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565697920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761681.883202:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565698048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761681.883230:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761681.883245:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761681/real 1789761681] req@ffff93f844d67480 x1876699565698048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761681.883259:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565698048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761681.883269:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761681.883272:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761681.883276:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565698048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761681.883283:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565698176:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761681.883300:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761681.883308:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761681/real 1789761681] req@ffff93f844d65880 x1876699565698176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761681.883316:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565698176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761681.883322:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761681.883325:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761681.883327:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565698176:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761681.883331:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565698304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761681.883342:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761681.883348:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761681/real 1789761681] req@ffff93f844d66680 x1876699565698304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761681.883355:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565698304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761681.883360:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761681.883362:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761681.883364:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565698304:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761681.946845:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761682.970823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761683.994794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761685.018816:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761686.042798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761687.002710:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761687.002732:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761687.002736:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761687.002744:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1621 00000100:00080000:3.0:1789761687.002751:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761687.002757:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761687.002788:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565698432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761687.002812:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761687.002815:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761687.002816:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761687.002819:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1621 00000100:00080000:3.0:1789761687.002822:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761687.002825:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761687.002833:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761687.002842:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565698432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761687.002844:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565698560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761687.002856:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761687.002858:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761687.002859:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761687.002861:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1621 00000100:00080000:3.0:1789761687.002863:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761687.002866:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761687.002869:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761687.002879:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565698688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761687.002886:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761687.002888:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761687.002889:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761687.002892:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1621 00000100:00080000:3.0:1789761687.002895:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761687.002898:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761687.002902:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761687.002905:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761687.002913:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565698816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761687.002953:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761687/real 1789761687] req@ffff93f844d64700 x1876699565698432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761687.002969:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565698432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761687.002979:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761687.002982:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761687.002987:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565698432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761687.003006:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565698560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761687.003027:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761687.003036:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761687/real 1789761687] req@ffff93f844d67b80 x1876699565698560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761687.003044:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565698560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761687.003050:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761687.003052:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761687.003054:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565698560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761687.003058:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565698688:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761687.003070:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761687.003078:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761687/real 1789761687] req@ffff93f844d66680 x1876699565698688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761687.003090:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565698688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761687.003105:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761687.003108:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761687.003112:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565698688:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761687.003119:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565698816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761687.003141:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761687.003153:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761687/real 1789761687] req@ffff93f844d65880 x1876699565698816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761687.003166:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565698816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761687.003175:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761687.003179:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761687.003182:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565698816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761687.066776:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761688.090821:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761689.114792:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761690.138806:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761691.162792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761692.122767:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761692.122781:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761692.122785:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761692.122794:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1627 00000100:00080000:0.0:1789761692.122802:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761692.122808:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761692.122841:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565698944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761692.122863:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761692.122866:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761692.122868:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761692.122871:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1627 00000100:00080000:0.0:1789761692.122873:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761692.122876:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761692.122884:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761692.122892:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565699072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761692.122905:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789761692.122907:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565698944:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761692.122908:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761692.122912:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761692.122918:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1627 00000100:00080000:0.0:1789761692.122920:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761692.122922:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761692.122925:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761692.122933:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565699200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761692.122940:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761692.122943:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761692.122944:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761692.122947:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1627 00000100:00080000:0.0:1789761692.122949:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761692.122951:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761692.122954:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761692.122960:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565699328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761692.122982:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761692.123028:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761692/real 1789761692] req@ffff93f850d41c00 x1876699565698944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761692.123044:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565698944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761692.123052:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761692.123056:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761692.123059:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565698944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761692.123078:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565699072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761692.123095:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761692.123104:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761692/real 1789761692] req@ffff93f850d40e00 x1876699565699072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761692.123112:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565699072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761692.123118:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761692.123121:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761692.123123:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565699072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761692.123127:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565699200:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761692.123137:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761692.123143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761692/real 1789761692] req@ffff93f850d40700 x1876699565699200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761692.123151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565699200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761692.123156:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761692.123158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761692.123160:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565699200:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761692.123163:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565699328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761692.123174:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761692.123180:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761692/real 1789761692] req@ffff93f850d43b80 x1876699565699328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761692.123188:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565699328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761692.123193:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761692.123195:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761692.123197:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565699328:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761692.186778:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761693.210783:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761694.234781:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761695.258800:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761696.282844:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761697.242943:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761697.242959:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761697.242966:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761697.242981:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1632 00000100:00080000:2.0:1789761697.242997:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761697.243007:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761697.243061:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565699456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761697.243096:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761697.243101:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761697.243104:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761697.243109:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1632 00000100:00080000:2.0:1789761697.243115:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761697.243122:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761697.243134:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761697.243151:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565699456:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761697.243154:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565699584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761697.243187:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761697.243192:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761697.243194:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761697.243202:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1632 00000100:00080000:2.0:1789761697.243209:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761697.243215:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761697.243221:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789761697.243243:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761697.243243:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565699712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761697.243260:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761697.243265:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761697.243268:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761697.243274:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1632 00000100:00080000:2.0:1789761697.243280:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761697.243286:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761697.243292:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761697.243310:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565699840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761697.243313:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761697/real 1789761697] req@ffff93f8517c2300 x1876699565699456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761697.243359:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565699456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761697.243371:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761697.243376:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761697.243380:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565699456:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761697.243417:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565699584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761697.243439:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761697.243453:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761697/real 1789761697] req@ffff93f8517c1f80 x1876699565699584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761697.243467:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565699584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761697.243476:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761697.243480:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761697.243483:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565699584:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761697.243490:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565699712:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761697.243515:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761697.243527:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761697/real 1789761697] req@ffff93f8517c3b80 x1876699565699712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761697.243538:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565699712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761697.243548:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761697.243550:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761697.243553:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565699712:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761697.243560:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565699840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761697.243583:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761697.243596:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761697/real 1789761697] req@ffff93f8517c0000 x1876699565699840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761697.243609:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565699840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761697.243618:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761697.243621:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761697.243623:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565699840:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761697.306774:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761698.330817:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761699.354798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761700.378789:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761701.402764:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761702.362755:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761702.362763:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761702.362767:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761702.362776:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1637 00000100:00080000:3.0:1789761702.362784:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761702.362790:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761702.362823:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565699968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761702.362849:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761702.362852:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761702.362853:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761702.362856:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1637 00000100:00080000:3.0:1789761702.362858:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761702.362861:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761702.362869:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761702.362877:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565700096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761702.362885:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761702.362888:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761702.362889:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761702.362893:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1637 00000100:00080000:3.0:1789761702.362896:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789761702.362897:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565699968:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761702.362899:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761702.362905:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761702.362912:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565700224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761702.362919:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761702.362921:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761702.362923:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761702.362925:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1637 00000100:00080000:3.0:1789761702.362928:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761702.362931:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761702.362934:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761702.362941:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565700352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761702.362970:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761702.363026:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761702/real 1789761702] req@ffff93f844d67480 x1876699565699968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761702.363046:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565699968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761702.363054:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761702.363057:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761702.363060:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565699968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761702.363080:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565700096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761702.363099:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761702.363108:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761702/real 1789761702] req@ffff93f844d67800 x1876699565700096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761702.363116:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565700096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761702.363122:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761702.363124:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761702.363126:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565700096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761702.363132:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565700224:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761702.363151:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761702.363163:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761702/real 1789761702] req@ffff93f844d67100 x1876699565700224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761702.363175:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565700224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761702.363183:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761702.363187:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761702.363191:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565700224:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761702.363197:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565700352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761702.363218:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761702.363230:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761702/real 1789761702] req@ffff93f844d66a00 x1876699565700352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761702.363255:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565700352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761702.363264:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761702.363267:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761702.363271:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565700352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761702.426849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761703.450765:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761704.474770:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761705.498797:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761706.522758:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761707.482739:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761707.482747:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761707.482751:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761707.482760:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1642 00000100:00080000:3.0:1789761707.482769:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761707.482775:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761707.482806:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565700480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761707.482827:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761707.482830:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761707.482831:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761707.482835:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1642 00000100:00080000:3.0:1789761707.482837:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761707.482840:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761707.482847:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761707.482861:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565700608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761707.482865:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565700480:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761707.482885:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761707.482889:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761707.482892:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761707.482898:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1642 00000100:00080000:3.0:1789761707.482903:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761707.482909:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761707.482915:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761707.482939:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565700736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761707.482940:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761707.482960:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761707.482965:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761707.482968:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761707.482975:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1642 00000100:00080000:3.0:1789761707.482989:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789761707.482996:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761707/real 1789761707] req@ffff93f844d64000 x1876699565700480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761707.482996:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761707.483008:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761707.483022:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565700480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761707.483035:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761707.483040:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761707.483046:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565700480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761707.483071:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565700608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761707.483081:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565700864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761707.483097:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761707.483115:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761707/real 1789761707] req@ffff93f844d66d80 x1876699565700608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761707.483128:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565700608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761707.483142:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761707.483146:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761707.483150:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565700608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761707.483156:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565700736:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761707.483174:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761707.483184:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761707/real 1789761707] req@ffff93f844d64e00 x1876699565700736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761707.483196:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565700736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761707.483204:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761707.483207:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761707.483210:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565700736:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761707.483229:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565700864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761707.483249:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761707.483262:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761707/real 1789761707] req@ffff93f844d65c00 x1876699565700864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761707.483274:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565700864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761707.483284:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761707.483288:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761707.483292:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565700864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761707.546835:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761708.570815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761709.594792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761710.618792:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761711.642792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761712.602736:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761712.602744:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761712.602749:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761712.602757:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1647 00000100:00080000:0.0:1789761712.602765:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761712.602771:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761712.602806:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565700992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761712.602833:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761712.602837:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761712.602839:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761712.602853:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1647 00000100:00080000:0.0:1789761712.602858:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761712.602861:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761712.602867:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565700992:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1789761712.602879:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761712.602899:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565701120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761712.602912:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761712.602917:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761712.602919:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761712.602923:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1647 00000100:00080000:0.0:1789761712.602927:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761712.602930:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761712.602936:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789761712.602936:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761712.602951:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565701248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761712.602960:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761712.602963:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761712.602964:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761712.602966:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1647 00000100:00080000:0.0:1789761712.602969:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761712.602971:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761712.602976:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761712.602984:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565701376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761712.602989:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761712/real 1789761712] req@ffff93f850d42300 x1876699565700992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761712.603019:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565700992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761712.603030:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761712.603034:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761712.603039:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565700992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761712.603061:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565701120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761712.603084:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761712.603098:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761712/real 1789761712] req@ffff93f850d40380 x1876699565701120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761712.603111:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565701120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761712.603121:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761712.603125:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761712.603129:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565701120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761712.603135:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565701248:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761712.603153:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761712.603163:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761712/real 1789761712] req@ffff93f850d43b80 x1876699565701248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761712.603175:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565701248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761712.603192:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761712.603195:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761712.603199:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565701248:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761712.603204:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565701376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761712.603220:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761712.603229:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761712/real 1789761712] req@ffff93f850d40700 x1876699565701376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761712.603241:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565701376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761712.603250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761712.603253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761712.603257:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565701376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761712.666774:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761713.690800:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761714.714764:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761715.738813:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761716.762796:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761717.722728:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761717.722743:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761717.722749:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761717.722762:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1652 00000100:00080000:3.0:1789761717.722774:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761717.722781:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761717.722821:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565701504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761717.722846:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761717.722849:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761717.722851:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761717.722855:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1652 00000100:00080000:3.0:1789761717.722858:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761717.722862:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761717.722870:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761717.722878:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565701504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761717.722882:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565701632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761717.722893:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761717.722895:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761717.722897:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761717.722901:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1652 00000100:00080000:3.0:1789761717.722904:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761717.722906:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761717.722910:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761717.722920:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565701760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761717.722927:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761717.722930:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761717.722931:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761717.722933:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1652 00000100:00080000:3.0:1789761717.722937:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761717.722940:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761717.722943:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761717.722954:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565701888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761717.723003:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565701632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761717.723028:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565701760:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761717.723045:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565701888:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761717.786799:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761718.810811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761719.834823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789761720.794989:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789761720.795042:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789761720.795060:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761720.795102:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761720.795116:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761720.795130:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789761720.795147:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761717/real 1789761720] req@ffff93f844d65f80 x1876699565701632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761720.795181:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565701632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761720.795198:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761720.795203:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761720.795208:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565701632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761720.795215:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761717/real 1789761720] req@ffff93f844d64a80 x1876699565701760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761720.795236:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565701760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761720.795246:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761720.795250:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761720.795254:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565701760:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761720.795262:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761717/real 1789761720] req@ffff93f844d64380 x1876699565701888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761720.795276:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565701888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761720.795285:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761720.795288:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761720.795290:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565701888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761720.795295:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761717/real 1789761720] req@ffff93f844d65500 x1876699565701504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761720.795306:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565701504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761720.795315:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761720.795319:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761720.795323:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565701504:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761720.858767:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761721.882836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761722.842732:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761722.842742:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761722.842747:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761722.842756:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1657 00000100:00080000:3.0:1789761722.842768:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761722.842777:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761722.842813:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565702016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761722.842842:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761722.842846:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761722.842847:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761722.842852:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1657 00000100:00080000:3.0:1789761722.842856:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761722.842860:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761722.842871:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761722.842885:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565702144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761722.842896:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761722.842906:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761722.842908:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761722.842913:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1657 00000100:00080000:3.0:1789761722.842917:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761722.842927:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761722.842929:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565702016:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789761722.842934:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761722.842962:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565702272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761722.842976:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761722.842981:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761722.842983:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761722.842987:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1657 00000100:00080000:3.0:1789761722.842991:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761722.842998:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761722.843004:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761722.843019:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565702400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761722.843056:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761722.843137:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761722/real 1789761722] req@ffff93f844d66300 x1876699565702016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761722.843160:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565702016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761722.843173:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761722.843178:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761722.843183:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565702016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761722.843194:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565702144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761722.843223:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761722.843240:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761722/real 1789761722] req@ffff93f844d65180 x1876699565702144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761722.843268:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565702144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761722.843277:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761722.843281:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761722.843283:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565702144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761722.843317:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565702272:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761722.843341:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761722.843355:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761722/real 1789761722] req@ffff93f844d65500 x1876699565702272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761722.843367:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565702272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761722.843376:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761722.843380:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761722.843383:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565702272:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761722.843390:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565702400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761722.843414:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761722.843427:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761722/real 1789761722] req@ffff93f844d64380 x1876699565702400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761722.843441:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565702400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761722.843463:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761722.843467:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761722.843471:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565702400:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761722.906885:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761723.930821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761724.954803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761725.978787:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761727.002839:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761727.962738:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761727.962749:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761727.962759:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761727.962771:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1662 00000100:00080000:3.0:1789761727.962779:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761727.962785:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761727.962824:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565702528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761727.962854:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761727.962857:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761727.962858:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761727.962862:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1662 00000100:00080000:3.0:1789761727.962866:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761727.962870:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761727.962879:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761727.962891:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565702656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761727.962900:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761727.962903:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761727.962905:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761727.962915:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1662 00000100:00100000:2.0:1789761727.962916:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565702528:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761727.962918:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761727.962928:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761727.962939:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761727.962947:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565702784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761727.962954:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761727.962958:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761727.962960:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761727.962962:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1662 00000100:00080000:3.0:1789761727.962965:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761727.962968:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761727.962971:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761727.962988:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565702912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761727.963024:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761727.963100:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761727/real 1789761727] req@ffff93f844d64a80 x1876699565702528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761727.963127:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565702528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761727.963150:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761727.963159:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761727.963164:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565702528:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761727.963199:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565702656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761727.963230:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761727.963248:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761727/real 1789761727] req@ffff93f844d65f80 x1876699565702656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761727.963263:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565702656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761727.963269:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761727.963272:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761727.963274:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565702656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761727.963280:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565702784:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761727.963298:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761727.963307:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761727/real 1789761727] req@ffff93f844d65c00 x1876699565702784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761727.963315:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565702784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761727.963325:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761727.963327:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761727.963330:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565702784:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761727.963333:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565702912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761727.963345:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761727.963351:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761727/real 1789761727] req@ffff93f844d64e00 x1876699565702912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761727.963358:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565702912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761727.963364:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761727.963366:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761727.963367:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565702912:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761728.026824:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761729.050776:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761730.074814:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761731.098842:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761732.122787:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761733.082786:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761733.082797:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761733.082802:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761733.082813:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1668 00000100:00080000:3.0:1789761733.082826:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761733.082833:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761733.082874:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565703040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761733.082898:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761733.082901:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761733.082903:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761733.082906:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1668 00000100:00080000:3.0:1789761733.082909:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761733.082914:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761733.082928:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761733.082944:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565703168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761733.082945:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565703040:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761733.082970:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761733.082972:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761733.082974:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761733.082980:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1668 00000100:00080000:3.0:1789761733.082982:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761733.082986:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761733.082991:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761733.083004:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565703296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761733.083012:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761733.083016:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761733.083018:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761733.083020:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1668 00000100:00080000:3.0:1789761733.083023:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761733.083025:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761733.083029:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761733.083034:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761733.083041:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565703424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761733.083101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761733/real 1789761733] req@ffff93f844d66d80 x1876699565703040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761733.083120:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565703040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761733.083129:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761733.083132:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761733.083134:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565703040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761733.083158:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565703168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761733.083177:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761733.083187:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761733/real 1789761733] req@ffff93f844d64000 x1876699565703168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761733.083195:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565703168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761733.083201:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761733.083203:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761733.083205:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565703168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761733.083209:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565703296:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761733.083221:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761733.083227:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761733/real 1789761733] req@ffff93f844d66a00 x1876699565703296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761733.083234:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565703296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761733.083239:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761733.083241:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761733.083243:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565703296:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761733.083247:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565703424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761733.083258:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761733.083264:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761733/real 1789761733] req@ffff93f844d67100 x1876699565703424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761733.083271:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565703424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761733.083276:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761733.083278:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761733.083280:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565703424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761733.146804:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761734.170767:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761735.194780:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761736.218793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761737.242874:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761738.202799:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761738.202814:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761738.202820:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761738.202831:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1673 00000100:00080000:2.0:1789761738.202842:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761738.202850:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761738.202886:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565703552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761738.202914:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761738.202917:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761738.202918:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761738.202921:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1673 00000100:00080000:2.0:1789761738.202924:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761738.202926:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761738.202934:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761738.202944:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565703680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761738.202957:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761738.202960:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761738.202961:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761738.202964:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1673 00000100:00080000:2.0:1789761738.202966:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789761738.202971:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565703552:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761738.202977:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761738.202983:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761738.202989:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565703808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761738.202997:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761738.203000:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761738.203001:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761738.203004:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1673 00000100:00080000:2.0:1789761738.203007:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761738.203009:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761738.203012:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761738.203020:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565703936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789761738.203044:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761738.203101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761738/real 1789761738] req@ffff93f8517c0000 x1876699565703552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761738.203119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565703552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761738.203132:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761738.203137:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761738.203142:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565703552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761738.203155:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565703680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761738.203179:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761738.203192:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761738/real 1789761738] req@ffff93f8517c3b80 x1876699565703680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761738.203205:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565703680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761738.203213:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761738.203217:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761738.203220:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565703680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761738.203244:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565703808:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761738.203265:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761738.203276:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761738/real 1789761738] req@ffff93f8517c1f80 x1876699565703808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761738.203288:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565703808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761738.203296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761738.203299:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761738.203302:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565703808:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761738.203307:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565703936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761738.203324:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761738.203351:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761738/real 1789761738] req@ffff93f8517c2300 x1876699565703936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761738.203364:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565703936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761738.203371:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761738.203375:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761738.203378:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565703936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761738.266796:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761739.290796:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761740.314761:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761741.338830:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761742.362807:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761743.322761:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761743.322774:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761743.322780:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761743.322795:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1678 00000100:00080000:2.0:1789761743.322810:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761743.322820:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761743.322867:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565704064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761743.322905:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761743.322911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761743.322913:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761743.322923:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1678 00000100:00080000:2.0:1789761743.322929:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761743.322934:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761743.322948:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761743.322963:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565704064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761743.322970:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565704192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761743.322988:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761743.322993:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761743.322995:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761743.323001:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1678 00000100:00080000:2.0:1789761743.323005:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761743.323011:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761743.323020:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761743.323038:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565704320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761743.323051:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761743.323054:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761743.323060:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761743.323063:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761743.323071:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1678 00000100:00080000:2.0:1789761743.323076:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761743.323082:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761743.323088:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761743.323102:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761743/real 1789761743] req@ffff93f8517c2680 x1876699565704064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761743.323105:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565704448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761743.323169:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565704064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761743.323186:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761743.323191:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761743.323196:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565704064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761743.323225:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565704192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761743.323260:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761743.323281:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761743/real 1789761743] req@ffff93f8517c0380 x1876699565704192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761743.323319:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565704192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761743.323331:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761743.323339:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761743.323346:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565704192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761743.323368:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565704320:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761743.323395:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761743.323409:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761743/real 1789761743] req@ffff93f8517c1500 x1876699565704320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761743.323423:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565704320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761743.323433:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761743.323437:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761743.323441:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565704320:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761743.323448:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565704448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761743.323470:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761743.323483:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761743/real 1789761743] req@ffff93f8517c0a80 x1876699565704448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761743.323496:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565704448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761743.323505:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761743.323509:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761743.323513:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565704448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761743.386856:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761744.410819:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761745.434784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761746.458799:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761747.482796:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761748.442757:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761748.442769:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761748.442773:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761748.442785:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1683 00000100:00080000:0.0:1789761748.442796:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761748.442804:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761748.442853:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565704576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761748.442878:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761748.442882:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761748.442883:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761748.442886:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1683 00000100:00080000:0.0:1789761748.442889:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761748.442891:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761748.442900:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761748.442910:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565704704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761748.442925:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761748.442930:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:1.0:1789761748.442930:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565704576:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761748.442938:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761748.442943:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1683 00000100:00080000:0.0:1789761748.442945:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761748.442948:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761748.442952:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761748.442961:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565704832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761748.442969:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761748.442972:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761748.442974:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761748.442977:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1683 00000100:00080000:0.0:1789761748.442991:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761748.442994:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761748.442997:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789761748.443004:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761748.443009:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565704960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761748.443051:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761748/real 1789761748] req@ffff93f850d40e00 x1876699565704576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761748.443069:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565704576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761748.443078:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761748.443081:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761748.443085:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565704576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761748.443109:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565704704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761748.443124:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761748.443133:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761748/real 1789761748] req@ffff93f850d41c00 x1876699565704704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761748.443152:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565704704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761748.443161:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761748.443164:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761748.443166:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565704704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761748.443171:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565704832:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761748.443186:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761748.443193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761748/real 1789761748] req@ffff93f850d42680 x1876699565704832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761748.443201:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565704832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761748.443212:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761748.443214:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761748.443216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565704832:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761748.443220:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565704960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761748.443231:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761748.443236:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761748/real 1789761748] req@ffff93f850d40000 x1876699565704960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761748.443244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565704960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761748.443250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761748.443252:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761748.443267:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565704960:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761748.506834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761749.530814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761750.554825:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761751.578793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761752.602886:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761753.562754:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761753.562766:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761753.562771:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761753.562783:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1688 00000100:00080000:0.0:1789761753.562794:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761753.562801:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761753.562848:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565705088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761753.562873:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761753.562876:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761753.562878:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761753.562882:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1688 00000100:00080000:0.0:1789761753.562887:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761753.562892:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761753.562902:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761753.562906:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565705088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761753.562920:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565705216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761753.562933:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761753.562937:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761753.562939:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761753.562952:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1688 00000100:00080000:0.0:1789761753.562964:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761753.562969:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1789761753.562974:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1789761753.562976:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761753.562992:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565705344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761753.563004:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761753.563008:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761753.563010:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761753.563015:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1688 00000100:00080000:0.0:1789761753.563020:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1789761753.563022:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761753/real 1789761753] req@ffff93f850d41180 x1876699565705088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761753.563027:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761753.563036:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761753.563048:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565705088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761753.563055:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565705472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761753.563060:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761753.563069:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761753.563076:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565705088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761753.563099:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565705216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761753.563118:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761753.563127:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761753/real 1789761753] req@ffff93f850d43100 x1876699565705216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761753.563136:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565705216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761753.563142:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761753.563144:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761753.563146:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565705216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761753.563150:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565705344:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761753.563162:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761753.563168:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761753/real 1789761753] req@ffff93f850d41f80 x1876699565705344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761753.563176:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565705344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761753.563184:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761753.563188:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761753.563192:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565705344:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761753.563198:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565705472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761753.563234:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761753.563249:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761753/real 1789761753] req@ffff93f850d41500 x1876699565705472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761753.563263:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565705472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761753.563272:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761753.563275:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761753.563279:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565705472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761753.626845:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761754.650823:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761755.674792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761756.698811:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761757.722806:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761758.682812:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761758.682827:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761758.682833:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761758.682849:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1693 00000100:00080000:3.0:1789761758.682862:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761758.682875:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761758.682925:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565705600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761758.682959:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761758.682968:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761758.682970:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761758.682974:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1693 00000100:00080000:3.0:1789761758.682979:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761758.682985:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761758.682997:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761758.683011:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565705600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761758.683016:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565705728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761758.683032:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761758.683036:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761758.683038:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761758.683043:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1693 00000100:00080000:3.0:1789761758.683048:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761758.683053:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761758.683060:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761758.683073:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565705856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761758.683083:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761758.683087:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761758.683093:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761758.683103:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761758.683110:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1693 00000100:00080000:3.0:1789761758.683115:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761758.683120:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761758.683126:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761758.683142:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565705984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761758.683145:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761758/real 1789761758] req@ffff93f844d67800 x1876699565705600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761758.683195:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565705600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761758.683209:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761758.683213:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761758.683217:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565705600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761758.683254:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565705728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761758.683290:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761758.683302:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761758/real 1789761758] req@ffff93f844d67480 x1876699565705728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761758.683312:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565705728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761758.683323:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761758.683326:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761758.683328:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565705728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761758.683332:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565705856:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761758.683348:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761758.683355:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761758/real 1789761758] req@ffff93f844d65880 x1876699565705856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761758.683377:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565705856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761758.683383:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761758.683385:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761758.683388:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565705856:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761758.683391:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565705984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761758.683425:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761758.683438:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761758/real 1789761758] req@ffff93f844d66680 x1876699565705984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761758.683450:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565705984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761758.683459:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761758.683463:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761758.683467:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565705984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761758.746817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761759.770802:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761760.794781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761761.818831:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761762.842777:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761763.802745:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761763.802757:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761763.802762:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761763.802774:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1698 00000100:00080000:3.0:1789761763.802785:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761763.802794:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761763.802833:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565706112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761763.802860:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761763.802864:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761763.802866:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761763.802876:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1698 00000100:00080000:3.0:1789761763.802881:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761763.802888:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761763.802898:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565706112:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789761763.802898:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761763.802918:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565706240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761763.802932:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761763.802936:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761763.802939:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761763.802944:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1698 00000100:00080000:3.0:1789761763.802949:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761763.802954:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761763.802960:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:1.0:1789761763.802970:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761763.802975:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565706368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761763.802989:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761763.802992:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761763.802994:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761763.802999:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1698 00000100:00080000:3.0:1789761763.803003:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761763.803007:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761763.803013:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761763.803015:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761763/real 1789761763] req@ffff93f844d67b80 x1876699565706112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761763.803030:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565706496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761763.803041:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565706112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761763.803055:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761763.803060:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761763.803065:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565706112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761763.803088:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565706240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761763.803114:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761763.803128:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761763/real 1789761763] req@ffff93f844d64700 x1876699565706240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761763.803158:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565706240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761763.803168:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761763.803171:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761763.803175:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565706240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761763.803183:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565706368:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761763.803207:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761763.803220:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761763/real 1789761763] req@ffff93f844d66680 x1876699565706368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761763.803233:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565706368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761763.803242:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761763.803245:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761763.803247:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565706368:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761763.803251:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565706496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761763.803267:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761763.803275:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761763/real 1789761763] req@ffff93f844d65880 x1876699565706496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761763.803283:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565706496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761763.803289:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761763.803291:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761763.803293:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565706496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761763.866803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761764.890757:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761765.914773:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761766.938780:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761767.962763:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761768.922732:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761768.922742:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761768.922746:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761768.922755:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1703 00000100:00080000:3.0:1789761768.922766:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761768.922774:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761768.922811:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565706624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761768.922836:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761768.922840:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761768.922841:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761768.922846:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1703 00000100:00080000:3.0:1789761768.922851:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761768.922856:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761768.922866:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761768.922870:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565706624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761768.922882:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565706752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761768.922895:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761768.922899:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761768.922901:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761768.922905:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1703 00000100:00080000:3.0:1789761768.922910:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761768.922914:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761768.922920:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:1.0:1789761768.922928:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761768.922936:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565706880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761768.922948:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761768.922952:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761768.922955:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761768.922961:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1703 00000100:00080000:3.0:1789761768.922966:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1789761768.922967:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761768/real 1789761768] req@ffff93f844d67480 x1876699565706624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761768.922973:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761768.922982:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761768.922998:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565706624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761768.922999:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565707008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761768.923034:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761768.923039:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761768.923043:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565706624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761768.923064:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565706752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761768.923081:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761768.923090:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761768/real 1789761768] req@ffff93f844d67800 x1876699565706752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761768.923114:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565706752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761768.923124:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761768.923128:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761768.923132:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565706752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761768.923139:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565706880:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761768.923174:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761768.923182:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761768/real 1789761768] req@ffff93f844d67100 x1876699565706880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761768.923191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565706880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761768.923196:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761768.923203:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761768.923205:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565706880:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761768.923209:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565707008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761768.923222:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761768.923228:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761768/real 1789761768] req@ffff93f844d66a00 x1876699565707008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761768.923235:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565707008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761768.923241:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761768.923242:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761768.923244:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565707008:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761768.986765:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761770.010807:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761771.034793:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761772.058764:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761773.082779:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761774.042729:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761774.042739:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761774.042745:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761774.042755:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1709 00000100:00080000:3.0:1789761774.042766:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761774.042773:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761774.042807:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565707136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761774.042834:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761774.042838:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761774.042840:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761774.042844:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1709 00000100:00080000:3.0:1789761774.042848:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761774.042852:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761774.042863:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761774.042873:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565707136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761774.042879:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565707264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761774.042891:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761774.042895:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761774.042897:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761774.042900:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1709 00000100:00080000:3.0:1789761774.042903:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761774.042908:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761774.042913:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761774.042925:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565707392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761774.042936:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:1.0:1789761774.042943:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761774.042944:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761774.042949:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761774.042955:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1709 00000100:00080000:3.0:1789761774.042960:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761774.042964:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761774.042969:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761774.042981:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565707520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761774.042991:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761774/real 1789761774] req@ffff93f844d64000 x1876699565707136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761774.043021:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565707136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761774.043033:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761774.043037:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761774.043041:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565707136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761774.043083:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565707264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761774.043109:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761774.043131:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761774/real 1789761774] req@ffff93f844d66d80 x1876699565707264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761774.043145:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565707264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761774.043155:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761774.043158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761774.043162:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565707264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761774.043169:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565707392:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761774.043190:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761774.043201:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761774/real 1789761774] req@ffff93f844d64e00 x1876699565707392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761774.043214:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565707392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761774.043223:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761774.043226:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761774.043230:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565707392:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761774.043236:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565707520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761774.043257:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761774.043268:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761774/real 1789761774] req@ffff93f844d65c00 x1876699565707520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761774.043281:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565707520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761774.043296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761774.043299:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761774.043303:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565707520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761774.106836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761775.130767:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761776.154828:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761777.178927:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761778.202831:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761779.162795:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761779.162804:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761779.162809:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761779.162819:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1714 00000100:00080000:3.0:1789761779.162830:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761779.162836:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761779.162870:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565707648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761779.162902:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761779.162905:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761779.162907:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761779.162910:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1714 00000100:00080000:3.0:1789761779.162913:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761779.162917:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761779.162926:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761779.162936:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565707776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761779.162952:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761779.162955:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761779.162957:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761779.162961:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1714 00000100:00080000:3.0:1789761779.162964:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1789761779.162968:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565707648:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761779.162975:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761779.162980:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761779.162992:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565707904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761779.163003:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761779.163006:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761779.163012:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761779.163016:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1714 00000100:00080000:3.0:1789761779.163020:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1789761779.163034:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761779.163044:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761779.163050:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761779.163062:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565708032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761779.163086:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761779/real 1789761779] req@ffff93f844d65f80 x1876699565707648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761779.163105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565707648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761779.163113:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761779.163116:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761779.163118:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565707648:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761779.163125:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565707776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761779.163143:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761779.163156:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761779/real 1789761779] req@ffff93f844d64a80 x1876699565707776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761779.163172:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565707776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761779.163180:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761779.163183:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761779.163187:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565707776:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761779.163221:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565707904:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761779.163249:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761779.163261:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761779/real 1789761779] req@ffff93f844d64380 x1876699565707904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761779.163271:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565707904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761779.163276:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761779.163280:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761779.163285:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565707904:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761779.163289:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565708032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761779.163302:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761779.163309:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761779/real 1789761779] req@ffff93f844d65500 x1876699565708032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761779.163405:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565708032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761779.163414:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761779.163418:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761779.163421:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565708032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761779.226791:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761780.250794:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761781.274801:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761782.298834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761783.322863:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761784.282768:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761784.282777:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761784.282781:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761784.282792:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1719 00000100:00080000:1.0:1789761784.282802:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761784.282809:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761784.282845:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565708160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761784.282871:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761784.282875:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761784.282877:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761784.282883:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1719 00000100:00080000:1.0:1789761784.282887:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761784.282891:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761784.282900:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761784.282914:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565708160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761784.282916:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565708288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761784.282937:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761784.282940:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761784.282942:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761784.282946:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1719 00000100:00080000:1.0:1789761784.282952:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761784.282956:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761784.282962:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761784.282977:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565708416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761784.283004:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761784.283008:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761784.283011:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761784.283014:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1719 00000100:00080000:1.0:1789761784.283020:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761784.283024:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761784.283029:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761784.283044:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565708544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761784.283064:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565708288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761784.283104:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565708416:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761784.283130:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565708544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761784.346784:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761785.370804:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761786.394793:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789761787.354885:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789761787.354922:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789761787.354935:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761787.354976:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761787.354992:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789761787.355006:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789761787.355025:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761784/real 1789761787] req@ffff93f844ccaa00 x1876699565708288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761787.355060:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565708288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761787.355074:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761787.355079:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761787.355083:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565708288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761787.355100:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761784/real 1789761787] req@ffff93f844cc8700 x1876699565708416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761787.355123:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565708416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761787.355131:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761787.355134:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761787.355138:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565708416:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761787.355146:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761784/real 1789761787] req@ffff93f844ccbb80 x1876699565708544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761787.355159:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565708544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761787.355166:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761787.355170:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761787.355173:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565708544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761787.355179:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761784/real 1789761787] req@ffff93f844cc8000 x1876699565708160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761787.355190:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565708160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761787.355197:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761787.355200:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761787.355203:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565708160:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761787.418857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761788.442787:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761789.402771:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761789.402779:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761789.402782:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761789.402789:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1724 00000100:00080000:3.0:1789761789.402797:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761789.402802:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761789.402827:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565708672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761789.402850:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761789.402858:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761789.402860:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761789.402864:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1724 00000100:00080000:3.0:1789761789.402868:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761789.402874:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761789.402884:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761789.402905:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565708672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761789.402905:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565708800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761789.402932:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761789.402937:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761789.402940:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761789.402946:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1724 00000100:00080000:3.0:1789761789.402963:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761789.402969:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761789.402976:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761789.402989:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565708928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761789.403000:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1789761789.403009:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761789.403011:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761789.403015:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761789.403021:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1724 00000100:00080000:3.0:1789761789.403026:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761789.403032:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761789.403037:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761789.403051:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565709056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761789.403155:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761789/real 1789761789] req@ffff93f844d65180 x1876699565708672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761789.403178:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565708672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761789.403190:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761789.403196:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761789.403200:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565708672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761789.403233:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565708800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761789.403275:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761789.403288:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761789/real 1789761789] req@ffff93f844d66300 x1876699565708800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761789.403300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565708800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761789.403308:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761789.403311:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761789.403314:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565708800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761789.403319:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565708928:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761789.403339:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761789.403348:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761789/real 1789761789] req@ffff93f844d65500 x1876699565708928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761789.403363:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565708928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761789.403369:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761789.403372:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761789.403375:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565708928:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761789.403380:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565709056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761789.403396:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761789.403407:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761789/real 1789761789] req@ffff93f844d64380 x1876699565709056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761789.403417:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565709056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761789.403423:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761789.403432:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761789.403434:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565709056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761789.466787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761790.490768:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761791.514845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761792.538812:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761793.562791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761794.522733:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761794.522748:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761794.522752:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761794.522761:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1729 00000100:00080000:2.0:1789761794.522770:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761794.522776:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761794.522810:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565709184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761794.522829:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761794.522832:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761794.522833:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761794.522836:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1729 00000100:00080000:2.0:1789761794.522838:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761794.522841:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761794.522848:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761794.522857:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565709312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761794.522863:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565709184:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761794.522869:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761794.522872:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761794.522873:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761794.522877:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1729 00000100:00080000:2.0:1789761794.522879:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761794.522882:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761794.522885:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761794.522893:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565709440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761794.522900:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761794.522903:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761794.522904:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761794.522906:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1729 00000100:00080000:2.0:1789761794.522926:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761794.522930:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761794.522933:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789761794.522941:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761794.522943:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565709568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761794.522981:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761794/real 1789761794] req@ffff93f8517c1180 x1876699565709184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761794.522995:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565709184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761794.523008:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761794.523011:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761794.523014:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565709184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761794.523034:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565709312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761794.523051:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761794.523060:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761794/real 1789761794] req@ffff93f8517c3100 x1876699565709312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761794.523067:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565709312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761794.523073:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761794.523075:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761794.523077:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565709312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761794.523081:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565709440:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761794.523092:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761794.523097:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761794/real 1789761794] req@ffff93f8517c2a00 x1876699565709440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761794.523105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565709440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761794.523110:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761794.523112:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761794.523114:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565709440:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761794.523117:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565709568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761794.523127:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761794.523133:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761794/real 1789761794] req@ffff93f8517c2d80 x1876699565709568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761794.523140:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565709568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761794.523145:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761794.523147:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761794.523149:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565709568:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761794.586816:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761795.610818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761796.634803:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761797.658801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761798.682785:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761799.642719:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761799.642727:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761799.642731:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761799.642738:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1734 00000100:00080000:2.0:1789761799.642747:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761799.642753:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761799.642797:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565709696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761799.642825:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761799.642830:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761799.642832:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761799.642838:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1734 00000100:00080000:2.0:1789761799.642844:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761799.642851:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761799.642860:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761799.642861:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565709696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761799.642888:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565709824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761799.642898:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761799.642900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761799.642902:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761799.642905:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1734 00000100:00080000:2.0:1789761799.642909:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761799.642911:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761799.642915:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761799.642924:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565709952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761799.642932:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761799.642936:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761799.642937:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761799.642939:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1734 00000800:00000100:0.0:1789761799.642944:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761799.642945:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761799.642950:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761799.642954:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761799.642962:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565710080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761799.642992:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761799/real 1789761799] req@ffff93f8517c0e00 x1876699565709696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761799.643009:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565709696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761799.643018:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761799.643023:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761799.643027:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565709696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761799.643046:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565709824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761799.643063:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761799.643071:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761799/real 1789761799] req@ffff93f8517c3480 x1876699565709824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761799.643080:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565709824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761799.643085:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761799.643088:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761799.643090:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565709824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761799.643093:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565709952:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761799.643104:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761799.643110:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761799/real 1789761799] req@ffff93f8517c3800 x1876699565709952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761799.643117:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565709952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761799.643122:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761799.643124:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761799.643126:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565709952:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761799.643129:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565710080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761799.643139:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761799.643144:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761799/real 1789761799] req@ffff93f8517c1880 x1876699565710080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761799.643151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565710080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761799.643160:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761799.643162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761799.643164:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565710080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761799.706889:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761800.730752:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761801.754791:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761802.778858:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761803.802796:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761804.762778:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761804.762789:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761804.762794:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761804.762806:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1739 00000100:00080000:0.0:1789761804.762818:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761804.762826:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761804.762885:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565710208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761804.762919:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761804.762922:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761804.762924:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761804.762927:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1739 00000100:00080000:0.0:1789761804.762931:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761804.762935:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761804.762945:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761804.762957:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565710336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761804.762965:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565710208:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761804.762969:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761804.762975:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761804.762978:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761804.762983:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1739 00000100:00080000:0.0:1789761804.762986:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761804.762990:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761804.762994:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761804.763005:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565710464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761804.763012:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761804.763015:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761804.763016:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761804.763020:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1739 00000100:00080000:0.0:1789761804.763022:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761804.763025:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761804.763028:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761804.763035:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565710592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761804.763046:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761804.763101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761804/real 1789761804] req@ffff93f850d43800 x1876699565710208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761804.763117:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565710208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761804.763126:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761804.763130:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761804.763133:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565710208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761804.763156:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565710336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761804.763173:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761804.763182:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761804/real 1789761804] req@ffff93f850d43480 x1876699565710336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761804.763191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565710336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761804.763197:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761804.763200:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761804.763202:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565710336:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761804.763206:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565710464:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761804.763218:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761804.763231:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761804/real 1789761804] req@ffff93f850d41500 x1876699565710464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761804.763239:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565710464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761804.763245:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761804.763247:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761804.763249:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565710464:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761804.763252:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565710592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761804.763265:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761804.763271:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761804/real 1789761804] req@ffff93f850d41f80 x1876699565710592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761804.763279:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565710592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761804.763284:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761804.763286:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761804.763288:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565710592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761804.826876:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761805.850805:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761806.874850:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761807.898832:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761808.922841:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761809.882759:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761809.882768:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761809.882771:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761809.882780:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1744 00000100:00080000:3.0:1789761809.882808:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761809.882816:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761809.882853:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565710720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761809.882878:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761809.882889:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761809.882891:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761809.882898:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1744 00000100:00080000:3.0:1789761809.882904:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761809.882909:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761809.882919:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761809.882924:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565710720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761809.882934:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565710848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761809.882944:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761809.882948:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761809.882950:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761809.882953:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1744 00000100:00080000:3.0:1789761809.882957:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761809.882959:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761809.882963:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761809.882973:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565710976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761809.882981:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761809.882983:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761809.882984:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761809.882986:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1744 00000100:00080000:3.0:1789761809.882990:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761809.882993:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761809.882996:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761809.883003:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565711104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761809.883011:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761809.883083:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761809/real 1789761809] req@ffff93f844d64a80 x1876699565710720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761809.883113:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565710720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761809.883124:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761809.883127:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761809.883131:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565710720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761809.883157:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565710848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761809.883180:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761809.883191:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761809/real 1789761809] req@ffff93f844d65f80 x1876699565710848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761809.883199:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565710848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761809.883205:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761809.883207:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761809.883210:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565710848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761809.883214:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565710976:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761809.883229:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761809.883236:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761809/real 1789761809] req@ffff93f844d65c00 x1876699565710976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761809.883244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565710976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761809.883250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761809.883252:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761809.883254:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565710976:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761809.883267:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565711104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761809.883290:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761809.883303:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761809/real 1789761809] req@ffff93f844d64e00 x1876699565711104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761809.883316:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565711104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761809.883323:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761809.883325:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761809.883328:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565711104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761809.946847:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761810.970912:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761811.994815:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761813.018858:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761814.042873:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761815.002729:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761815.002740:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761815.002746:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761815.002771:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1749 00000100:00080000:3.0:1789761815.002781:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761815.002789:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761815.002825:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565711232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761815.002848:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761815.002851:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761815.002853:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761815.002856:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1749 00000100:00080000:3.0:1789761815.002858:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761815.002861:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761815.002870:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761815.002879:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565711360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761815.002889:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565711232:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761815.002890:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761815.002895:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761815.002898:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761815.002903:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1749 00000100:00080000:3.0:1789761815.002906:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761815.002908:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761815.002912:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761815.002921:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565711488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761815.002935:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761815.002939:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761815.002941:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761815.002944:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1749 00000100:00080000:3.0:1789761815.002958:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761815.002961:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761815.002965:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761815.002978:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565711616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761815.002981:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761815.003041:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761815/real 1789761815] req@ffff93f844d66d80 x1876699565711232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761815.003062:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565711232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761815.003074:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761815.003078:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761815.003083:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565711232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761815.003119:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565711360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761815.003149:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761815.003165:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761815/real 1789761815] req@ffff93f844d64000 x1876699565711360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761815.003180:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565711360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761815.003190:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761815.003194:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761815.003198:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565711360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761815.003204:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565711488:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761815.003224:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761815.003234:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761815/real 1789761815] req@ffff93f844d66a00 x1876699565711488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761815.003243:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565711488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761815.003250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761815.003253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761815.003256:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565711488:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761815.003261:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565711616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761815.003275:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761815.003284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761815/real 1789761815] req@ffff93f844d67100 x1876699565711616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761815.003292:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565711616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761815.003297:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761815.003300:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761815.003302:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565711616:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761815.066825:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761816.090851:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761817.114800:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761818.138814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761819.162830:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761820.122758:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761820.122766:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761820.122770:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761820.122778:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1755 00000100:00080000:1.0:1789761820.122786:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761820.122792:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761820.122825:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565711744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761820.122846:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761820.122849:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761820.122851:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761820.122854:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1755 00000100:00080000:1.0:1789761820.122856:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761820.122858:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761820.122867:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761820.122876:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565711872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761820.122885:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789761820.122887:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565711744:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761820.122888:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761820.122892:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761820.122898:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1755 00000100:00080000:1.0:1789761820.122900:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761820.122902:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761820.122905:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761820.122914:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565712000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761820.122922:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761820.122924:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761820.122926:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761820.122928:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1755 00000100:00080000:1.0:1789761820.122931:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761820.122933:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761820.122936:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761820.122943:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565712128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761820.122950:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761820.122992:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761820/real 1789761820] req@ffff93f844cc9880 x1876699565711744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761820.123007:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565711744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761820.123015:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761820.123018:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761820.123020:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565711744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761820.123039:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565711872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761820.123054:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761820.123062:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761820/real 1789761820] req@ffff93f844cc8380 x1876699565711872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761820.123070:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565711872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761820.123076:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761820.123078:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761820.123080:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565711872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761820.123084:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565712000:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761820.123094:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761820.123100:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761820/real 1789761820] req@ffff93f844cc9180 x1876699565712000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761820.123107:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565712000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761820.123115:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761820.123117:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761820.123119:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565712000:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761820.123123:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565712128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761820.123137:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761820.123143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761820/real 1789761820] req@ffff93f844cc8a80 x1876699565712128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761820.123151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565712128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761820.123160:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761820.123162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761820.123164:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565712128:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761820.186773:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761821.210987:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761822.234863:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761823.258879:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761824.282867:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761825.242782:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761825.242791:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761825.242796:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761825.242807:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1760 00000100:00080000:3.0:1789761825.242818:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761825.242825:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761825.242862:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565712256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761825.242890:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761825.242894:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761825.242896:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761825.242901:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1760 00000100:00080000:3.0:1789761825.242905:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761825.242910:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761825.242920:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761825.242934:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565712256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761825.242937:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565712384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761825.242956:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761825.242961:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761825.242963:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761825.242970:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1760 00000100:00080000:3.0:1789761825.242981:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761825.242989:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761825.242995:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761825.243010:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565712512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761825.243021:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761825.243034:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761825.243038:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761825.243041:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761825.243048:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1760 00000100:00080000:3.0:1789761825.243053:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761825.243059:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761825.243065:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761825.243078:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565712640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761825.243101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761825/real 1789761825] req@ffff93f844d67800 x1876699565712256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761825.243130:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565712256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761825.243141:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761825.243144:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761825.243153:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565712256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761825.243190:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565712384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761825.243218:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761825.243233:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761825/real 1789761825] req@ffff93f844d67480 x1876699565712384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761825.243242:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565712384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761825.243249:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761825.243251:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761825.243254:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565712384:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761825.243259:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565712512:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761825.243276:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761825.243283:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761825/real 1789761825] req@ffff93f844d65880 x1876699565712512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761825.243291:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565712512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761825.243297:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761825.243299:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761825.243301:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565712512:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761825.243305:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565712640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761825.243317:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761825.243323:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761825/real 1789761825] req@ffff93f844d66680 x1876699565712640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761825.243329:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565712640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761825.243335:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761825.243337:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761825.243339:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565712640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761825.306815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761826.330889:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761827.354787:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761828.378823:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761829.402862:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761830.363065:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761830.363076:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761830.363081:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761830.363091:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1765 00000100:00080000:0.0:1789761830.363101:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761830.363109:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761830.363150:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565712768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761830.363187:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761830.363191:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761830.363193:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761830.363198:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1765 00000100:00080000:0.0:1789761830.363202:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761830.363206:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761830.363215:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761830.363227:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565712896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761830.363235:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761830.363238:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761830.363240:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789761830.363243:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565712768:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761830.363245:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1765 00000100:00080000:0.0:1789761830.363250:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761830.363255:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761830.363258:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761830.363266:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565713024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761830.363273:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761830.363276:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761830.363277:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761830.363279:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1765 00000100:00080000:0.0:1789761830.363291:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761830.363294:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761830.363297:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761830.363305:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565713152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761830.363315:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761830.363358:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761830/real 1789761830] req@ffff93f850d43100 x1876699565712768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761830.363374:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565712768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761830.363382:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761830.363386:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761830.363389:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565712768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761830.363408:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565712896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761830.363423:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761830.363433:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761830/real 1789761830] req@ffff93f850d41180 x1876699565712896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761830.363441:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565712896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761830.363446:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761830.363449:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761830.363451:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565712896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761830.363454:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565713024:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761830.363465:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761830.363472:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761830/real 1789761830] req@ffff93f850d40000 x1876699565713024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761830.363479:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565713024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761830.363489:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761830.363491:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761830.363493:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565713024:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761830.363496:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565713152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761830.363506:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761830.363512:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761830/real 1789761830] req@ffff93f850d42680 x1876699565713152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761830.363520:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565713152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761830.363525:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761830.363527:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761830.363528:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565713152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761830.426829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761831.450788:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761832.474780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761833.498773:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761834.522804:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761835.482758:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761835.482769:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761835.482774:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761835.482785:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1770 00000100:00080000:3.0:1789761835.482797:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761835.482805:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761835.482849:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565713280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761835.482875:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761835.482878:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761835.482880:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761835.482884:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1770 00000100:00080000:3.0:1789761835.482887:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761835.482890:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761835.482900:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761835.482910:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565713408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761835.482920:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565713280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761835.482923:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761835.482938:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761835.482941:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761835.482946:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1770 00000100:00080000:3.0:1789761835.482948:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761835.482950:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761835.482955:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761835.482967:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565713536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761835.482975:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761835.482977:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761835.482979:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761835.482983:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1770 00000100:00080000:3.0:1789761835.483002:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789761835.483008:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761835.483011:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761835.483020:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761835.483039:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565713664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761835.483086:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761835/real 1789761835] req@ffff93f844d64700 x1876699565713280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761835.483112:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565713280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761835.483125:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761835.483130:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761835.483134:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565713280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761835.483161:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565713408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761835.483182:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761835.483192:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761835/real 1789761835] req@ffff93f844d67b80 x1876699565713408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761835.483201:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565713408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761835.483207:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761835.483209:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761835.483211:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565713408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761835.483215:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565713536:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761835.483226:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761835.483232:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761835/real 1789761835] req@ffff93f844d66680 x1876699565713536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761835.483239:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565713536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761835.483245:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761835.483246:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761835.483248:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565713536:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761835.483254:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565713664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761835.483275:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761835.483289:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761835/real 1789761835] req@ffff93f844d65880 x1876699565713664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761835.483302:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565713664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761835.483311:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761835.483314:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761835.483317:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565713664:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761835.546800:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761836.570828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761837.594767:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761838.618788:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761839.642846:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761840.602720:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761840.602728:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761840.602732:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761840.602740:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1775 00000100:00080000:3.0:1789761840.602748:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761840.602754:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761840.602785:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565713792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761840.602808:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761840.602811:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761840.602812:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761840.602815:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1775 00000100:00080000:3.0:1789761840.602817:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761840.602820:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761840.602827:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761840.602845:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565713792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761840.602845:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565713920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761840.602853:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761840.602856:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761840.602857:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761840.602859:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1775 00000100:00080000:3.0:1789761840.602861:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761840.602863:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761840.602866:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761840.602872:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565714048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761840.602878:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761840.602881:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761840.602882:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761840.602884:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1775 00000100:00080000:3.0:1789761840.602885:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761840.602890:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761840.602898:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761840.602918:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565714176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761840.602954:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761840.603006:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761840/real 1789761840] req@ffff93f844d67480 x1876699565713792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761840.603021:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565713792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761840.603037:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761840.603040:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761840.603044:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565713792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761840.603070:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565713920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761840.603089:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761840.603099:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761840/real 1789761840] req@ffff93f844d67800 x1876699565713920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761840.603108:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565713920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761840.603119:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761840.603121:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761840.603123:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565713920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761840.603127:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565714048:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761840.603147:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761840.603155:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761840/real 1789761840] req@ffff93f844d67100 x1876699565714048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761840.603163:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565714048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761840.603170:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761840.603174:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761840.603177:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565714048:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761840.603183:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565714176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761840.603205:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761840.603218:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761840/real 1789761840] req@ffff93f844d66a00 x1876699565714176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761840.603229:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565714176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761840.603235:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761840.603237:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761840.603240:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565714176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761840.666794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761841.690817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761842.714792:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761843.738832:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761844.762802:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761845.722769:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761845.722778:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761845.722782:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761845.722791:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1780 00000100:00080000:2.0:1789761845.722799:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761845.722805:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761845.722846:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565714304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761845.722877:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761845.722881:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761845.722883:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761845.722887:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1780 00000100:00080000:2.0:1789761845.722890:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761845.722894:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761845.722902:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761845.722915:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565714432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761845.722920:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565714304:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761845.722928:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761845.722932:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761845.722934:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761845.722937:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1780 00000100:00080000:2.0:1789761845.722939:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761845.722943:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761845.722946:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761845.722955:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565714560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761845.722962:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761845.722964:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761845.722966:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761845.722968:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1780 00000100:00080000:2.0:1789761845.722972:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761845.722975:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761845.722977:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789761845.722983:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761845.722987:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565714688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761845.723029:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761845/real 1789761845] req@ffff93f8517c1880 x1876699565714304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761845.723044:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565714304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761845.723052:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761845.723056:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761845.723059:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565714304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761845.723085:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565714432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761845.723101:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761845.723110:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761845/real 1789761845] req@ffff93f8517c3800 x1876699565714432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761845.723118:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565714432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761845.723124:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761845.723126:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761845.723129:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565714432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761845.723133:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565714560:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761845.723144:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761845.723150:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761845/real 1789761845] req@ffff93f8517c3480 x1876699565714560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761845.723157:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565714560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761845.723162:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761845.723164:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761845.723166:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565714560:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761845.723170:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565714688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761845.723180:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761845.723190:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761845/real 1789761845] req@ffff93f8517c0e00 x1876699565714688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761845.723203:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565714688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761845.723219:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761845.723223:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761845.723227:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565714688:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761845.786794:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761846.810782:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761847.834771:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761848.858770:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761849.882781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761850.842771:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761850.842780:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761850.842785:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761850.842796:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1785 00000100:00080000:3.0:1789761850.842806:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761850.842822:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761850.842861:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565714816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761850.842891:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761850.842896:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761850.842898:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761850.842905:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1785 00000100:00080000:3.0:1789761850.842911:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761850.842917:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761850.842921:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565714816:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789761850.842930:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761850.842950:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565714944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761850.842964:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789761850.843027:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565714944:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1789761850.843050:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761850.843053:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761850.843060:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1785 00000100:00080000:3.0:1789761850.843066:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761850.843072:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761850.843079:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761850.843097:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565715072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761850.843130:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761850.843134:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761850.843136:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761850.843142:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1785 00000100:00080000:3.0:1789761850.843149:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789761850.843155:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565715072:192.168.203.147@tcp:8:kworker.0 00000100:00080000:3.0:1789761850.843156:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761850.843168:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761850.843185:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565715200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761850.843215:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565715200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761850.906752:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761851.930848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761852.954821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789761853.914870:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789761853.914901:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789761853.914914:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761853.914945:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761853.914951:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761853.914957:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789761853.914984:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761850/real 1789761853] req@ffff93f844d65c00 x1876699565715200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761853.915003:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565715200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761853.915019:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761853.915025:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761853.915029:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565715200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761853.915036:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761850/real 1789761853] req@ffff93f844d64e00 x1876699565715072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761853.915048:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565715072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761853.915063:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761853.915066:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761853.915070:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565715072:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761853.915077:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761850/real 1789761853] req@ffff93f844d66d80 x1876699565714944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761853.915088:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565714944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761853.915097:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761853.915100:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761853.915110:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565714944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761853.915117:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761850/real 1789761853] req@ffff93f844d64000 x1876699565714816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761853.915131:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565714816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761853.915140:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761853.915143:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761853.915146:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565714816:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761853.978843:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761855.002785:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761855.962737:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761855.962748:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761855.962752:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761855.962762:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1790 00000100:00080000:3.0:1789761855.962772:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761855.962778:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761855.962813:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565715328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761855.962852:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761855.962856:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761855.962858:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761855.962863:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1790 00000100:00080000:3.0:1789761855.962867:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761855.962872:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761855.962882:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761855.962899:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565715456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761855.962901:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565715328:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761855.962915:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761855.962919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761855.962920:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761855.962923:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1790 00000100:00080000:3.0:1789761855.962927:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761855.962929:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761855.962933:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761855.962944:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565715584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761855.962956:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761855.962959:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761855.962962:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761855.962966:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1790 00000100:00080000:3.0:1789761855.962971:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761855.962976:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761855.962982:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761855.962985:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761855.962999:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565715712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761855.963047:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761855/real 1789761855] req@ffff93f844d65f80 x1876699565715328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761855.963066:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565715328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761855.963078:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761855.963082:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761855.963086:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565715328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761855.963112:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565715456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761855.963135:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761855.963148:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761855/real 1789761855] req@ffff93f844d64a80 x1876699565715456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761855.963161:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565715456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761855.963169:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761855.963172:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761855.963176:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565715456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761855.963183:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565715584:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761855.963203:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761855.963214:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761855/real 1789761855] req@ffff93f844d64380 x1876699565715584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761855.963226:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565715584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761855.963234:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761855.963237:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761855.963240:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565715584:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761855.963246:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565715712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761855.963265:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761855.963275:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761855/real 1789761855] req@ffff93f844d65500 x1876699565715712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761855.963287:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565715712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761855.963296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761855.963299:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761855.963304:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565715712:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761856.026841:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761857.050778:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761858.074795:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761859.098799:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761860.122800:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761861.082755:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761861.082763:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761861.082768:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761861.082776:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1796 00000100:00080000:3.0:1789761861.082788:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761861.082794:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761861.082827:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565715840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761861.082849:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761861.082851:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761861.082853:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761861.082855:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1796 00000100:00080000:3.0:1789761861.082858:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761861.082860:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761861.082868:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761861.082882:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565715840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761861.082882:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565715968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761861.082901:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761861.082904:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761861.082905:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761861.082908:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1796 00000100:00080000:3.0:1789761861.082911:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761861.082913:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761861.082917:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761861.082927:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565716096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761861.082935:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761861.082937:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761861.082939:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761861.082941:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1796 00000100:00080000:3.0:1789761861.082944:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789761861.082948:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761861.082952:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761861.082956:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761861.082965:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565716224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761861.083034:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761861/real 1789761861] req@ffff93f844d66300 x1876699565715840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761861.083060:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565715840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761861.083069:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761861.083072:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761861.083075:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565715840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761861.083096:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565715968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761861.083115:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761861.083124:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761861/real 1789761861] req@ffff93f844d65180 x1876699565715968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761861.083132:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565715968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761861.083138:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761861.083140:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761861.083142:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565715968:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761861.083146:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565716096:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761861.083158:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761861.083166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761861/real 1789761861] req@ffff93f844d65500 x1876699565716096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761861.083174:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565716096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761861.083183:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761861.083185:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761861.083187:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565716096:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761861.083191:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565716224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761861.083201:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761861.083206:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761861/real 1789761861] req@ffff93f844d64380 x1876699565716224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761861.083213:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565716224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761861.083218:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761861.083220:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761861.083222:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565716224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761861.146840:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761862.170820:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761863.194778:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761864.218791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761865.242801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761866.202771:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761866.202781:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761866.202817:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761866.202833:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1801 00000100:00080000:3.0:1789761866.202869:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761866.202881:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761866.202939:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565716352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761866.202996:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761866.203001:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761866.203014:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761866.203022:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1801 00000100:00080000:3.0:1789761866.203030:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761866.203036:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761866.203047:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761866.203053:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565716352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761866.203066:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565716480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761866.203078:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761866.203082:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761866.203084:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761866.203088:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1801 00000100:00080000:3.0:1789761866.203092:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761866.203095:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761866.203101:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761866.203112:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565716608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761866.203122:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761866.203125:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761866.203127:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761866.203130:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1801 00000100:00080000:3.0:1789761866.203143:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761866.203147:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761866.203155:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761866.203164:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761866.203179:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565716736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761866.203240:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761866/real 1789761866] req@ffff93f844d64a80 x1876699565716352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761866.203266:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565716352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761866.203280:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761866.203286:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761866.203290:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565716352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761866.203325:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565716480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761866.203352:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761866.203367:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761866/real 1789761866] req@ffff93f844d65f80 x1876699565716480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761866.203395:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565716480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761866.203405:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761866.203409:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761866.203413:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565716480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761866.203421:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565716608:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761866.203446:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761866.203459:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761866/real 1789761866] req@ffff93f844d64000 x1876699565716608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761866.203472:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565716608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761866.203480:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761866.203484:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761866.203487:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565716608:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761866.203493:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565716736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761866.203513:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761866.203523:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761866/real 1789761866] req@ffff93f844d66d80 x1876699565716736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761866.203535:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565716736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761866.203543:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761866.203546:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761866.203549:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565716736:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761866.266851:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761867.290823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761868.314876:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761869.338855:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761870.362775:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761871.322724:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761871.322735:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761871.322739:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761871.322748:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1806 00000100:00080000:3.0:1789761871.322756:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761871.322762:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761871.322796:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565716864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761871.322833:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761871.322836:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761871.322837:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761871.322840:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1806 00000100:00080000:3.0:1789761871.322843:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761871.322846:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761871.322865:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761871.322876:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565716864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761871.322877:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565716992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761871.322890:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761871.322893:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761871.322894:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761871.322897:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1806 00000100:00080000:3.0:1789761871.322900:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761871.322902:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761871.322905:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761871.322914:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565717120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761871.322922:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761871.322926:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761871.322928:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761871.322932:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1806 00000100:00080000:3.0:1789761871.322937:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789761871.322942:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761871.322946:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761871.322953:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761871.322967:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565717248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761871.322997:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761871/real 1789761871] req@ffff93f844d64e00 x1876699565716864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761871.323019:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565716864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761871.323043:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761871.323047:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761871.323050:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565716864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761871.323073:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565716992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761871.323092:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761871.323101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761871/real 1789761871] req@ffff93f844d65c00 x1876699565716992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761871.323110:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565716992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761871.323120:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761871.323123:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761871.323127:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565716992:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761871.323133:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565717120:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761871.323152:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761871.323164:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761871/real 1789761871] req@ffff93f844d66a00 x1876699565717120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761871.323177:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565717120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761871.323194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761871.323196:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761871.323198:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565717120:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761871.323202:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565717248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761871.323217:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761871.323224:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761871/real 1789761871] req@ffff93f844d67100 x1876699565717248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761871.323232:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565717248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761871.323237:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761871.323239:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761871.323241:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565717248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761871.386887:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761872.410802:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761873.434785:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761874.458795:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761875.482825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761876.442773:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761876.442781:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761876.442785:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761876.442793:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1811 00000100:00080000:3.0:1789761876.442802:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761876.442808:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761876.442842:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565717376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761876.442863:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761876.442866:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761876.442867:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761876.442870:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1811 00000100:00080000:3.0:1789761876.442872:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761876.442874:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761876.442882:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761876.442892:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565717504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761876.442912:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789761876.442917:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565717376:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761876.442917:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761876.442921:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761876.442931:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1811 00000100:00080000:3.0:1789761876.442942:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761876.442947:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761876.442954:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761876.442969:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565717632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761876.442977:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761876.442980:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761876.442981:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789761876.442986:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761876.442990:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1811 00000100:00080000:3.0:1789761876.442995:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761876.442999:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761876.443003:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761876.443011:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565717760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761876.443038:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761876/real 1789761876] req@ffff93f844d67800 x1876699565717376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761876.443053:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565717376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761876.443061:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761876.443065:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761876.443069:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565717376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761876.443087:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565717504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761876.443103:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761876.443111:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761876/real 1789761876] req@ffff93f844d67480 x1876699565717504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761876.443119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565717504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761876.443125:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761876.443127:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761876.443129:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565717504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761876.443133:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565717632:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761876.443143:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761876.443150:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761876/real 1789761876] req@ffff93f844d65880 x1876699565717632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761876.443157:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565717632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761876.443162:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761876.443164:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761876.443166:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565717632:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761876.443169:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565717760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761876.443179:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761876.443185:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761876/real 1789761876] req@ffff93f844d66680 x1876699565717760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761876.443192:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565717760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761876.443197:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761876.443199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761876.443200:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565717760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761876.506824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761877.530828:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761878.554775:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761879.578922:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761880.602981:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761881.562764:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761881.562772:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761881.562776:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761881.562784:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1816 00000100:00080000:0.0:1789761881.562792:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761881.562797:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761881.562830:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565717888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761881.562852:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761881.562854:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761881.562856:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761881.562858:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1816 00000100:00080000:0.0:1789761881.562861:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761881.562864:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761881.562871:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761881.562880:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565718016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761881.562888:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761881.562891:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:2.0:1789761881.562891:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565717888:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761881.562894:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761881.562901:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1816 00000100:00080000:0.0:1789761881.562904:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761881.562906:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761881.562909:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761881.562918:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565718144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761881.562924:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761881.562926:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761881.562928:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761881.562931:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1816 00000100:00080000:0.0:1789761881.562933:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761881.562936:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761881.562939:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761881.562945:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565718272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761881.562959:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761881.563007:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761881/real 1789761881] req@ffff93f850d41c00 x1876699565717888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761881.563021:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565717888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761881.563030:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761881.563034:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761881.563037:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565717888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761881.563056:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565718016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761881.563076:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761881.563085:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761881/real 1789761881] req@ffff93f850d40e00 x1876699565718016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761881.563093:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565718016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761881.563098:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761881.563101:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761881.563103:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565718016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761881.563106:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565718144:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761881.563117:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761881.563123:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761881/real 1789761881] req@ffff93f850d40700 x1876699565718144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761881.563130:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565718144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761881.563135:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761881.563137:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761881.563139:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565718144:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761881.563143:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565718272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761881.563153:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761881.563159:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761881/real 1789761881] req@ffff93f850d43b80 x1876699565718272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761881.563166:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565718272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761881.563171:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761881.563173:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761881.563174:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565718272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761881.626790:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761882.650837:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761883.674754:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761884.698845:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761885.722813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761886.682755:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761886.682766:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761886.682770:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761886.682781:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1821 00000100:00080000:3.0:1789761886.682789:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761886.682797:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761886.682838:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565718400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761886.682863:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761886.682868:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761886.682869:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761886.682874:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1821 00000100:00080000:3.0:1789761886.682878:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761886.682883:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761886.682893:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761886.682907:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565718528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761886.682917:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565718400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761886.682924:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761886.682930:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761886.682932:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761886.682938:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1821 00000100:00080000:3.0:1789761886.682942:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789761886.682987:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761886.683012:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761886.683020:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761886.683021:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761886/real 1789761886] req@ffff93f844d67b80 x1876699565718400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789761886.683043:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565718656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761886.683044:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565718400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761886.683059:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761886.683065:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789761886.683065:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761886.683070:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761886.683071:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1789761886.683081:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1821 00000100:00100000:2.0:1789761886.683082:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565718400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761886.683089:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761886.683095:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761886.683100:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761886.683111:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565718528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789761886.683112:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565718784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761886.683147:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761886.683159:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761886/real 1789761886] req@ffff93f844d64700 x1876699565718528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761886.683168:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565718528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761886.683175:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761886.683177:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761886.683179:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565718528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761886.683184:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565718656:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761886.683197:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761886.683203:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761886/real 1789761886] req@ffff93f844d66680 x1876699565718656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761886.683211:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565718656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761886.683216:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761886.683218:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761886.683236:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565718656:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761886.683250:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565718784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761886.683272:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761886.683281:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761886/real 1789761886] req@ffff93f844d65880 x1876699565718784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761886.683290:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565718784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761886.683300:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761886.683303:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761886.683305:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565718784:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761886.746782:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761887.770784:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761888.794802:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761889.818790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761890.842800:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761891.802757:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761891.802765:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761891.802769:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761891.802778:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1826 00000100:00080000:3.0:1789761891.802787:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761891.802793:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761891.802827:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565718912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761891.802850:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761891.802853:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761891.802854:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761891.802857:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1826 00000100:00080000:3.0:1789761891.802859:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761891.802862:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761891.802870:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761891.802879:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565718912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761891.802880:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565719040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761891.802892:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761891.802894:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761891.802895:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761891.802898:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1826 00000100:00080000:3.0:1789761891.802901:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761891.802903:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761891.802912:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761891.802921:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565719168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761891.802928:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761891.802931:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761891.802932:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761891.802938:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1826 00000800:00000100:2.0:1789761891.802941:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761891.802943:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761891.802947:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761891.802950:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761891.802958:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565719296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761891.802999:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761891/real 1789761891] req@ffff93f844d67480 x1876699565718912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761891.803016:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565718912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761891.803024:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761891.803029:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761891.803032:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565718912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761891.803051:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565719040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761891.803069:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761891.803077:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761891/real 1789761891] req@ffff93f844d67800 x1876699565719040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761891.803085:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565719040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761891.803091:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761891.803093:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761891.803095:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565719040:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761891.803100:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565719168:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761891.803111:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761891.803117:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761891/real 1789761891] req@ffff93f844d67100 x1876699565719168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761891.803124:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565719168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761891.803130:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761891.803132:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761891.803134:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565719168:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761891.803137:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565719296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761891.803156:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761891.803166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761891/real 1789761891] req@ffff93f844d66a00 x1876699565719296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761891.803174:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565719296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761891.803179:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761891.803195:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761891.803197:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565719296:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761891.866789:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761892.890846:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761893.914784:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761894.938862:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761895.962779:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761896.922740:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761896.922751:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761896.922755:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761896.922766:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1831 00000100:00080000:3.0:1789761896.922776:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761896.922784:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761896.922822:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565719424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761896.922848:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761896.922852:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761896.922855:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761896.922859:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1831 00000100:00080000:3.0:1789761896.922864:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761896.922869:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761896.922879:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761896.922893:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565719552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761896.922905:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761896.922908:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761896.922910:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761896.922914:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1831 00000100:00080000:3.0:1789761896.922918:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761896.922923:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761896.922931:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761896.922944:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565719680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761896.922955:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761896.922959:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761896.922962:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761896.922966:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1831 00000100:00080000:3.0:1789761896.922971:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761896.922976:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761896.922983:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761896.922984:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565719424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761896.923000:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565719808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761896.923055:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761896.923109:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761896/real 1789761896] req@ffff93f844d65c00 x1876699565719424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761896.923131:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565719424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761896.923156:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761896.923162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761896.923167:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565719424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761896.923176:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565719552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761896.923202:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761896.923214:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761896/real 1789761896] req@ffff93f844d64e00 x1876699565719552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761896.923223:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565719552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761896.923232:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761896.923236:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761896.923239:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565719552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761896.923246:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565719680:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761896.923267:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761896.923280:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761896/real 1789761896] req@ffff93f844d66d80 x1876699565719680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761896.923293:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565719680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761896.923304:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761896.923307:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761896.923309:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565719680:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761896.923336:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565719808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761896.923354:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761896.923365:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761896/real 1789761896] req@ffff93f844d64000 x1876699565719808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761896.923376:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565719808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761896.923384:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761896.923387:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761896.923390:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565719808:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761896.986813:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761898.010794:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761899.034792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761900.058824:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761901.082854:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761902.042745:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761902.042757:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761902.042761:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761902.042772:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1837 00000100:00080000:3.0:1789761902.042781:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761902.042789:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761902.042828:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565719936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761902.042857:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761902.042868:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761902.042870:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761902.042875:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1837 00000100:00080000:3.0:1789761902.042879:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761902.042884:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761902.042895:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761902.042909:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565719936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761902.042912:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565720064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761902.042927:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761902.042931:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761902.042933:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761902.042938:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1837 00000100:00080000:3.0:1789761902.042942:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761902.042946:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761902.042952:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761902.042965:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565720192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761902.042976:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1789761902.042977:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761902.042983:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761902.042985:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761902.042990:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1837 00000100:00080000:3.0:1789761902.042994:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761902.042999:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761902.043004:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761902.043015:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565720320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761902.043033:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761902/real 1789761902] req@ffff93f844d65f80 x1876699565719936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761902.043053:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565719936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761902.043065:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761902.043069:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761902.043074:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565719936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761902.043100:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565720064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761902.043137:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761902.043148:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761902/real 1789761902] req@ffff93f844d64a80 x1876699565720064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761902.043158:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565720064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761902.043168:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761902.043171:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761902.043175:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565720064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761902.043182:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565720192:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761902.043204:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761902.043217:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761902/real 1789761902] req@ffff93f844d64380 x1876699565720192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761902.043226:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565720192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761902.043232:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761902.043234:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761902.043236:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565720192:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761902.043241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565720320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761902.043255:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761902.043261:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761902/real 1789761902] req@ffff93f844d65500 x1876699565720320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761902.043269:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565720320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761902.043274:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761902.043276:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761902.043278:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565720320:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761902.106777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761903.130767:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761904.154780:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761905.178789:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761906.202814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761907.162708:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761907.162716:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761907.162720:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761907.162729:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1842 00000100:00080000:3.0:1789761907.162739:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761907.162745:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761907.162778:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565720448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761907.162801:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761907.162803:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761907.162805:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761907.162808:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1842 00000100:00080000:3.0:1789761907.162810:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761907.162814:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761907.162822:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761907.162830:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565720448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761907.162834:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565720576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761907.162847:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761907.162851:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761907.162853:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761907.162857:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1842 00000100:00080000:3.0:1789761907.162861:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761907.162865:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761907.162870:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761907.162885:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565720704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761907.162897:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761907.162901:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761907.162903:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761907.162908:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1842 00000100:00080000:3.0:1789761907.162912:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789761907.162915:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789761907.162921:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761907.162928:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761907.162943:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565720832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761907.162978:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761907/real 1789761907] req@ffff93f844d65180 x1876699565720448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761907.162993:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565720448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761907.163001:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761907.163006:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761907.163010:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565720448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761907.163028:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565720576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761907.163043:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761907.163052:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761907/real 1789761907] req@ffff93f844d66300 x1876699565720576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761907.163060:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565720576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761907.163087:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761907.163089:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761907.163092:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565720576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761907.163097:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565720704:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761907.163119:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761907.163131:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761907/real 1789761907] req@ffff93f844d65500 x1876699565720704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761907.163144:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565720704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761907.163154:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761907.163157:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761907.163159:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565720704:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761907.163163:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565720832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761907.163179:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761907.163187:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761907/real 1789761907] req@ffff93f844d64380 x1876699565720832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761907.163194:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565720832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761907.163200:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761907.163202:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761907.163204:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565720832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761907.226780:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761908.250895:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761909.274896:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761910.298821:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761911.322931:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761912.282751:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761912.282761:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761912.282765:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761912.282775:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1847 00000100:00080000:0.0:1789761912.282786:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761912.282793:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761912.282831:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565720960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761912.282857:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761912.282861:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761912.282863:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761912.282866:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1847 00000100:00080000:0.0:1789761912.282870:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761912.282874:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761912.282885:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761912.282895:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565721088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761912.282906:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761912.282910:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:2.0:1789761912.282912:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565720960:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761912.282914:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761912.282928:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1847 00000100:00080000:0.0:1789761912.282931:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761912.282934:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761912.282941:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761912.282965:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565721216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761912.282990:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761912.282995:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761912.282997:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761912.283004:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1847 00000100:00080000:0.0:1789761912.283010:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761912.283014:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761912.283022:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761912.283049:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761912.283056:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565721344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761912.283134:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761912/real 1789761912] req@ffff93f850d40380 x1876699565720960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761912.283161:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565720960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761912.283176:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761912.283182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761912.283187:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565720960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761912.283220:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565721088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761912.283250:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761912.283266:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761912/real 1789761912] req@ffff93f850d42300 x1876699565721088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761912.283280:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565721088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761912.283289:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761912.283293:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761912.283297:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565721088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761912.283305:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565721216:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761912.283327:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761912.283337:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761912/real 1789761912] req@ffff93f850d43b80 x1876699565721216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761912.283346:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565721216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761912.283352:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761912.283354:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761912.283356:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565721216:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761912.283361:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565721344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761912.283374:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761912.283381:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761912/real 1789761912] req@ffff93f850d40700 x1876699565721344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761912.283389:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565721344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761912.283394:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761912.283396:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761912.283397:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565721344:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761912.346960:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761913.370775:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761914.394805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761915.418806:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761916.442772:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761917.402802:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761917.402812:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761917.402834:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761917.402857:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1852 00000100:00080000:1.0:1789761917.402868:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761917.402877:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761917.402948:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565721472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761917.402976:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761917.402980:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761917.402982:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761917.402986:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1852 00000100:00080000:1.0:1789761917.403032:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761917.403038:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761917.403047:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761917.403059:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565721600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761917.403080:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761917.403084:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761917.403087:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789761917.403088:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565721472:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761917.403092:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1852 00000100:00080000:1.0:1789761917.403098:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761917.403102:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761917.403106:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761917.403115:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565721728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761917.403129:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761917.403132:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761917.403133:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761917.403136:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1852 00000100:00080000:1.0:1789761917.403138:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761917.403141:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761917.403145:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761917.403152:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565721856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761917.403226:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565721600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761917.403267:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565721728:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761917.403310:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565721856:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761917.466777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761918.490759:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761919.514833:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789761920.474895:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789761920.474936:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789761920.474962:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761920.475014:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761920.475024:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789761920.475031:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789761920.475078:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761917/real 1789761920] req@ffff93f844cca300 x1876699565721600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761920.475106:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565721600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761920.475119:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761920.475124:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761920.475129:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565721600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761920.475137:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761917/real 1789761920] req@ffff93f844cc8a80 x1876699565721728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761920.475151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565721728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761920.475160:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761920.475164:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761920.475167:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565721728:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761920.475174:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761917/real 1789761920] req@ffff93f844cc9180 x1876699565721856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761920.475186:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565721856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761920.475194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761920.475197:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761920.475201:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565721856:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761920.475208:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761917/real 1789761920] req@ffff93f844cc9f80 x1876699565721472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761920.475220:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565721472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761920.475228:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761920.475232:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761920.475236:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565721472:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789761920.538784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761921.562818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761922.522779:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761922.522791:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761922.522797:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761922.522808:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1857 00000100:00080000:2.0:1789761922.522819:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761922.522829:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761922.522874:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565721984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761922.522928:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761922.522933:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761922.522935:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761922.522939:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1857 00000100:00080000:2.0:1789761922.522944:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761922.522962:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761922.522974:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761922.522979:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565721984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761922.522993:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565722112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761922.523008:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761922.523011:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761922.523013:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761922.523016:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1857 00000100:00080000:2.0:1789761922.523019:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761922.523022:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761922.523026:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761922.523038:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565722240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761922.523046:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761922.523050:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761922.523054:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1789761922.523055:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761922.523062:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1857 00000100:00080000:2.0:1789761922.523068:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761922.523073:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761922.523078:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761922.523091:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565722368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761922.523103:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761922/real 1789761922] req@ffff93f8517c2d80 x1876699565721984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761922.523124:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565721984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761922.523136:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761922.523141:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761922.523145:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565721984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761922.523174:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565722112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761922.523201:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761922.523215:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761922/real 1789761922] req@ffff93f8517c2a00 x1876699565722112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761922.523229:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565722112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761922.523240:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761922.523243:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761922.523246:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565722112:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761922.523252:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565722240:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761922.523273:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761922.523284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761922/real 1789761922] req@ffff93f8517c3100 x1876699565722240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761922.523297:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565722240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761922.523305:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761922.523309:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761922.523312:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565722240:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761922.523318:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565722368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761922.523337:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761922.523349:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761922/real 1789761922] req@ffff93f8517c1180 x1876699565722368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761922.523359:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565722368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761922.523372:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761922.523375:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761922.523378:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565722368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761922.586808:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761923.610940:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761924.634796:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761925.658786:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761926.682769:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761927.642766:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761927.642774:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761927.642779:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761927.642789:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1862 00000100:00080000:2.0:1789761927.642798:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761927.642804:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761927.642863:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565722496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761927.642906:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761927.642910:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761927.642927:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761927.642948:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1862 00000100:00080000:2.0:1789761927.642952:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761927.642956:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761927.642963:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565722496:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789761927.642964:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761927.642980:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565722624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761927.642989:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761927.642992:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761927.642993:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761927.642996:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1862 00000100:00080000:2.0:1789761927.643018:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761927.643021:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761927.643025:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761927.643037:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565722752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761927.643045:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761927.643053:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761927.643057:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761927.643059:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761927.643064:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1862 00000100:00080000:2.0:1789761927.643068:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761927.643070:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761927.643075:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761927.643083:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565722880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761927.643097:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761927/real 1789761927] req@ffff93f8517c0a80 x1876699565722496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761927.643116:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565722496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761927.643125:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761927.643128:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761927.643131:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565722496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761927.643151:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565722624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761927.643169:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761927.643178:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761927/real 1789761927] req@ffff93f8517c1500 x1876699565722624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761927.643187:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565722624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761927.643194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761927.643196:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761927.643199:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565722624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761927.643204:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565722752:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761927.643217:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761927.643224:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761927/real 1789761927] req@ffff93f8517c0380 x1876699565722752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761927.643232:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565722752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761927.643239:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761927.643241:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761927.643243:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565722752:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761927.643247:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565722880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761927.643258:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761927.643265:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761927/real 1789761927] req@ffff93f8517c2680 x1876699565722880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761927.643273:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565722880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761927.643279:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761927.643281:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761927.643283:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565722880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761927.706894:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761928.730813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761929.754849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761930.778828:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761931.802873:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761932.762851:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761932.762862:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761932.762867:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761932.762910:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1867 00000100:00080000:0.0:1789761932.762924:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761932.762933:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761932.762989:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565723008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761932.763025:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761932.763028:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761932.763030:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761932.763034:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1867 00000100:00080000:0.0:1789761932.763037:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761932.763042:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761932.763053:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761932.763083:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565723136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761932.763094:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761932.763100:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761932.763102:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789761932.763102:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565723008:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761932.763117:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1867 00000100:00080000:0.0:1789761932.763121:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761932.763126:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761932.763131:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761932.763147:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565723264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761932.763156:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761932.763160:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761932.763161:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761932.763166:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1867 00000100:00080000:0.0:1789761932.763183:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761932.763187:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761932.763191:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789761932.763193:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761932.763212:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565723392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761932.763248:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761932/real 1789761932] req@ffff93f850d40e00 x1876699565723008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761932.763263:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565723008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761932.763271:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761932.763275:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761932.763277:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565723008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761932.763299:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565723136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761932.763315:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761932.763323:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761932/real 1789761932] req@ffff93f850d41c00 x1876699565723136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761932.763331:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565723136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761932.763337:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761932.763339:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761932.763341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565723136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761932.763345:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565723264:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761932.763365:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761932.763371:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761932/real 1789761932] req@ffff93f850d42680 x1876699565723264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761932.763379:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565723264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761932.763389:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761932.763391:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761932.763393:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565723264:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761932.763397:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565723392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761932.763407:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761932.763413:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761932/real 1789761932] req@ffff93f850d40000 x1876699565723392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761932.763420:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565723392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761932.763426:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761932.763428:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761932.763430:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565723392:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761932.826858:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761933.850865:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761934.874838:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761935.898871:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761936.922791:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761937.882743:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761937.882751:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761937.882760:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761937.882769:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1872 00000100:00080000:1.0:1789761937.882777:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761937.882783:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761937.882817:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565723520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761937.882848:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761937.882853:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761937.882854:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761937.882858:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1872 00000100:00080000:1.0:1789761937.882861:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761937.882865:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761937.882874:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761937.882880:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565723520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761937.882886:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565723648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761937.882896:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761937.882898:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761937.882899:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761937.882901:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1872 00000100:00080000:1.0:1789761937.882904:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761937.882906:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761937.882909:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761937.882917:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565723776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761937.882924:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761937.882926:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761937.882928:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761937.882931:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1872 00000100:00080000:1.0:1789761937.882933:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761937.882936:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761937.882941:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761937.882949:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565723904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761937.882953:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761937.883003:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761937/real 1789761937] req@ffff93f844cc8380 x1876699565723520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761937.883020:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565723520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761937.883028:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761937.883032:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761937.883035:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565723520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761937.883054:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565723648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761937.883068:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761937.883076:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761937/real 1789761937] req@ffff93f844cc9880 x1876699565723648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761937.883085:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565723648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761937.883091:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761937.883093:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761937.883096:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565723648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761937.883105:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565723776:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761937.883116:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761937.883122:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761937/real 1789761937] req@ffff93f844cc8000 x1876699565723776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761937.883129:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565723776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761937.883134:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761937.883136:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761937.883138:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565723776:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761937.883141:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565723904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761937.883160:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761937.883166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761937/real 1789761937] req@ffff93f844ccbb80 x1876699565723904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761937.883173:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565723904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761937.883178:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761937.883180:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761937.883183:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565723904:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761937.946779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761938.970819:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761939.994767:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761941.018893:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761942.042778:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761943.002785:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761943.002809:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761943.002814:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761943.002822:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1877 00000100:00080000:1.0:1789761943.002830:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761943.002862:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761943.002894:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565724032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761943.002916:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761943.002919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761943.002920:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761943.002923:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1877 00000100:00080000:1.0:1789761943.002925:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761943.002928:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761943.002935:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761943.002945:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565724160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761943.002952:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761943.002956:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761943.002958:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789761943.002958:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565724032:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761943.002966:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1877 00000100:00080000:1.0:1789761943.002969:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761943.002972:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761943.002975:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761943.002983:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565724288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761943.002991:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761943.002994:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761943.002995:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761943.002998:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1877 00000100:00080000:1.0:1789761943.003001:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761943.003003:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761943.003006:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761943.003012:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565724416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761943.003022:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761943.003087:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761943/real 1789761943] req@ffff93f844cc8700 x1876699565724032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761943.003108:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565724032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761943.003121:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761943.003142:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761943.003147:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565724032:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761943.003173:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565724160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761943.003195:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761943.003206:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761943/real 1789761943] req@ffff93f844ccaa00 x1876699565724160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761943.003217:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565724160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761943.003224:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761943.003226:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761943.003229:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565724160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761943.003233:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565724288:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761943.003256:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761943.003266:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761943/real 1789761943] req@ffff93f844ccb800 x1876699565724288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761943.003275:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565724288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761943.003282:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761943.003284:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761943.003287:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565724288:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761943.003291:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565724416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761943.003304:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761943.003311:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761943/real 1789761943] req@ffff93f844ccb480 x1876699565724416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761943.003320:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565724416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761943.003325:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761943.003328:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761943.003330:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565724416:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761943.066814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761944.090760:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761945.114779:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761946.138803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761947.162859:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761948.122738:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761948.122750:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761948.122755:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761948.122784:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1883 00000100:00080000:1.0:1789761948.122795:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761948.122802:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761948.122847:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565724544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761948.122872:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761948.122878:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761948.122880:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761948.122883:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1883 00000100:00080000:1.0:1789761948.122886:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761948.122889:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761948.122898:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761948.122908:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565724672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761948.122916:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761948.122919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761948.122921:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789761948.122926:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565724544:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761948.122927:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1883 00000100:00080000:1.0:1789761948.122932:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761948.122937:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761948.122940:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761948.122948:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565724800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761948.122955:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761948.122958:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761948.122959:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761948.122962:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1883 00000100:00080000:1.0:1789761948.122964:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761948.122967:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761948.122971:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761948.122979:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565724928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761948.123003:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761948.123054:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761948/real 1789761948] req@ffff93f844ccb100 x1876699565724544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761948.123077:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565724544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761948.123091:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761948.123096:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761948.123100:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565724544:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761948.123133:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565724672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761948.123154:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761948.123164:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761948/real 1789761948] req@ffff93f844cc9c00 x1876699565724672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761948.123173:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565724672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761948.123179:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761948.123182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761948.123184:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565724672:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761948.123187:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565724800:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761948.123199:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761948.123206:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761948/real 1789761948] req@ffff93f844cca680 x1876699565724800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761948.123214:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565724800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761948.123219:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761948.123221:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761948.123223:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565724800:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761948.123227:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565724928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761948.123237:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761948.123243:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761948/real 1789761948] req@ffff93f844cc8e00 x1876699565724928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761948.123250:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565724928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761948.123255:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761948.123257:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761948.123258:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565724928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761948.186851:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761949.210825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761950.234832:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761951.258853:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761952.282787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761953.242746:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761953.242757:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761953.242762:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761953.242772:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1888 00000100:00080000:0.0:1789761953.242783:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761953.242789:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761953.242823:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565725056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761953.242846:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761953.242849:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761953.242850:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761953.242853:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1888 00000100:00080000:0.0:1789761953.242856:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761953.242859:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761953.242868:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761953.242877:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565725184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761953.242885:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565725056:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761953.242888:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761953.242894:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761953.242896:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761953.242900:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1888 00000100:00080000:0.0:1789761953.242902:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761953.242904:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761953.242907:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761953.242915:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565725312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761953.242923:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761953.242926:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761953.242927:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761953.242930:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1888 00000100:00080000:0.0:1789761953.242932:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761953.242935:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761953.242937:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761953.242943:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565725440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761953.242962:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761953.243011:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761953/real 1789761953] req@ffff93f850d41180 x1876699565725056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761953.243029:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565725056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761953.243040:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761953.243045:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761953.243050:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565725056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761953.243075:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565725184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761953.243095:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761953.243106:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761953/real 1789761953] req@ffff93f850d43100 x1876699565725184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761953.243115:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565725184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761953.243121:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761953.243123:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761953.243126:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565725184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761953.243130:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565725312:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761953.243141:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761953.243156:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761953/real 1789761953] req@ffff93f850d41f80 x1876699565725312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761953.243167:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565725312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761953.243172:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761953.243174:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761953.243177:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565725312:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761953.243180:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565725440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761953.243192:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761953.243199:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761953/real 1789761953] req@ffff93f850d41500 x1876699565725440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761953.243206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565725440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761953.243216:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761953.243218:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761953.243220:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565725440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761953.306822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761954.330775:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761955.354797:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761956.378801:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761957.402831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761958.362798:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761958.362814:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761958.362821:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761958.362834:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1893 00000100:00080000:0.0:1789761958.362847:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761958.362854:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761958.362899:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565725568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761958.362928:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761958.362932:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761958.362933:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761958.362936:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1893 00000100:00080000:0.0:1789761958.362939:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761958.362943:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761958.362953:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761958.362962:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565725696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761958.362971:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761958.362975:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761958.362978:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789761958.362978:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565725568:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761958.362986:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1893 00000100:00080000:0.0:1789761958.362989:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761958.362991:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761958.362994:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761958.363003:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565725824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761958.363011:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761958.363014:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761958.363015:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761958.363018:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1893 00000100:00080000:0.0:1789761958.363020:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761958.363023:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761958.363025:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761958.363033:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565725952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761958.363068:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761958.363114:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761958/real 1789761958] req@ffff93f850d43480 x1876699565725568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761958.363134:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565725568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761958.363149:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761958.363153:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761958.363157:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565725568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761958.363182:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565725696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761958.363205:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761958.363216:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761958/real 1789761958] req@ffff93f850d43800 x1876699565725696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761958.363226:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565725696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761958.363233:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761958.363241:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761958.363245:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565725696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761958.363251:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565725824:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761958.363269:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761958.363285:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761958/real 1789761958] req@ffff93f850d41500 x1876699565725824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761958.363300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565725824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761958.363311:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761958.363317:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761958.363323:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565725824:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761958.363330:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565725952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761958.363356:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761958.363371:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761958/real 1789761958] req@ffff93f850d41f80 x1876699565725952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761958.363389:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565725952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761958.363396:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761958.363399:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761958.363403:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565725952:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761958.426787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761959.450827:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761960.474843:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761961.498796:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761962.522846:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789761963.483729:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761963.483738:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789761963.483742:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761963.483750:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1898 00000100:00080000:0.0:1789761963.483758:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761963.483765:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789761963.483797:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565726080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761963.483818:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761963.483821:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761963.483822:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761963.483825:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1898 00000100:00080000:0.0:1789761963.483827:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761963.483830:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761963.483839:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761963.483849:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565726208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761963.483852:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565726080:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789761963.483865:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761963.483868:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789761963.483869:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761963.483872:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1898 00000100:00080000:0.0:1789761963.483874:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761963.483877:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761963.483881:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789761963.483890:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565726336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789761963.483898:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789761963.483900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789761963.483901:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789761963.483904:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1898 00000100:00080000:0.0:1789761963.483906:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789761963.483908:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789761963.483911:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761963.483917:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565726464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789761963.483931:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761963.483978:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761963/real 1789761963] req@ffff93f850d43100 x1876699565726080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761963.484000:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565726080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761963.484008:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761963.484011:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761963.484014:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565726080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761963.484034:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565726208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761963.484053:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761963.484064:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761963/real 1789761963] req@ffff93f850d41180 x1876699565726208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761963.484078:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565726208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761963.484087:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761963.484090:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761963.484094:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565726208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761963.484101:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565726336:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761963.484130:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761963.484139:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761963/real 1789761963] req@ffff93f850d40000 x1876699565726336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761963.484148:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565726336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761963.484158:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761963.484160:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761963.484163:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565726336:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761963.484166:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565726464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761963.484178:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761963.484184:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761963/real 1789761963] req@ffff93f850d42680 x1876699565726464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761963.484191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565726464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761963.484196:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761963.484198:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761963.484200:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565726464:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761963.546786:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761964.570859:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761965.594799:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761966.618842:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761967.642895:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761968.602747:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761968.602755:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761968.602759:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761968.602766:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1903 00000100:00080000:2.0:1789761968.602774:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761968.602779:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761968.602813:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565726592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761968.602835:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761968.602838:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761968.602840:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761968.602843:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1903 00000100:00080000:2.0:1789761968.602845:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761968.602849:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761968.602856:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761968.602869:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565726592:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761968.602870:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565726720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761968.602887:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761968.602889:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761968.602890:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761968.602892:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1903 00000100:00080000:2.0:1789761968.602894:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761968.602897:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761968.602901:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761968.602915:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565726848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761968.602926:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761968.602930:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761968.602933:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761968.602937:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1903 00000100:00080000:2.0:1789761968.602942:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1789761968.602947:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789761968.602951:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761968.602959:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761968.602978:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565726976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761968.602996:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761968/real 1789761968] req@ffff93f8517c2300 x1876699565726592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761968.603011:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565726592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761968.603019:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761968.603021:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761968.603024:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565726592:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761968.603044:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565726720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761968.603060:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761968.603069:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761968/real 1789761968] req@ffff93f8517c1f80 x1876699565726720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761968.603077:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565726720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761968.603085:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761968.603089:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761968.603093:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565726720:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761968.603099:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565726848:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761968.603121:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761968.603133:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761968/real 1789761968] req@ffff93f8517c3b80 x1876699565726848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761968.603146:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565726848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761968.603155:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761968.603159:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761968.603162:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565726848:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761968.603166:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565726976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761968.603182:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761968.603190:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761968/real 1789761968] req@ffff93f8517c0000 x1876699565726976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761968.603197:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565726976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761968.603202:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761968.603205:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761968.603207:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565726976:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761968.666823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761969.690783:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789761970.714813:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761971.738770:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761972.762847:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761973.722768:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761973.722779:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761973.722784:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761973.722794:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1908 00000100:00080000:2.0:1789761973.722807:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761973.722817:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761973.722868:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565727104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761973.722893:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761973.722896:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761973.722897:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761973.722901:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1908 00000100:00080000:2.0:1789761973.722904:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761973.722907:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761973.722916:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761973.722926:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565727232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761973.722934:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761973.722939:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:1.0:1789761973.722941:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565727104:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789761973.722941:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761973.722946:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1908 00000100:00080000:2.0:1789761973.722953:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761973.722956:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761973.722960:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761973.722966:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565727360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761973.722974:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761973.722976:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761973.722978:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761973.722981:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1908 00000100:00080000:2.0:1789761973.722983:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761973.722985:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761973.722988:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761973.722996:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565727488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761973.723039:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761973.723093:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761973/real 1789761973] req@ffff93f8517c0000 x1876699565727104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761973.723115:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565727104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789761998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761973.723124:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761973.723128:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761973.723132:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565727104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761973.723157:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565727232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761973.723174:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761973.723184:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761973/real 1789761973] req@ffff93f8517c3b80 x1876699565727232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761973.723192:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565727232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761973.723198:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761973.723200:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761973.723202:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565727232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761973.723206:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565727360:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761973.723217:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761973.723223:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761973/real 1789761973] req@ffff93f8517c1f80 x1876699565727360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761973.723231:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565727360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789761998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761973.723241:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761973.723243:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761973.723245:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565727360:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761973.723248:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565727488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761973.723258:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761973.723263:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761973/real 1789761973] req@ffff93f8517c2300 x1876699565727488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761973.723271:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565727488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789761998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761973.723276:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761973.723278:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761973.723280:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565727488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789761973.786780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761974.810789:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761975.834777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761976.858784:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761977.882877:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761978.842756:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761978.842765:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761978.842769:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761978.842778:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1913 00000100:00080000:3.0:1789761978.842787:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761978.842795:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761978.842835:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565727616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761978.842868:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761978.842871:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761978.842873:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761978.842876:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1913 00000100:00080000:3.0:1789761978.842879:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761978.842882:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761978.842890:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761978.842902:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565727744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761978.842909:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761978.842912:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761978.842914:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761978.842916:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1913 00000100:00080000:3.0:1789761978.842919:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761978.842921:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761978.842924:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761978.842928:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565727616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761978.842934:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565727872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761978.842943:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761978.842945:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761978.842946:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761978.842949:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1913 00000100:00080000:3.0:1789761978.842950:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761978.842953:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761978.842956:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761978.842962:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565728000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789761978.843001:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761978.843051:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761978/real 1789761978] req@ffff93f844d64a80 x1876699565727616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761978.843072:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565727616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761978.843084:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761978.843088:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761978.843092:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565727616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789761978.843100:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565727744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761978.843124:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761978.843137:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761978/real 1789761978] req@ffff93f844d65f80 x1876699565727744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761978.843150:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565727744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761978.843159:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761978.843163:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761978.843166:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565727744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761978.843193:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565727872:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789761978.843212:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761978.843223:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761978/real 1789761978] req@ffff93f844d64000 x1876699565727872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761978.843235:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565727872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761978.843243:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761978.843247:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761978.843250:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565727872:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761978.843256:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565728000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789761978.843275:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789761978.843288:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761978/real 1789761978] req@ffff93f844d66d80 x1876699565728000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789761978.843301:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565728000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789761978.843310:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789761978.843313:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789761978.843317:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565728000:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761978.906813:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761979.930808:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761980.954766:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761981.978837:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761983.002768:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789761983.962722:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761983.962732:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789761983.962735:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761983.962745:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1918 00000100:00080000:2.0:1789761983.962762:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761983.962771:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789761983.962804:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565728128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761983.962825:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761983.962831:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761983.962833:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761983.962837:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1918 00000100:00080000:2.0:1789761983.962842:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761983.962847:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761983.962858:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761983.962871:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565728128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761983.962882:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565728256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761983.962892:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761983.962894:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789761983.962896:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761983.962899:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1918 00000100:00080000:2.0:1789761983.962902:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761983.962905:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761983.962910:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789761983.962920:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565728384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789761983.962927:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789761983.962929:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789761983.962930:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789761983.962933:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1918 00000100:00080000:2.0:1789761983.962936:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789761983.962940:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789761983.962946:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789761983.962959:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565728512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789761983.962973:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565728256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789761983.962998:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565728384:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789761983.963015:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565728512:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761984.026794:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761985.050757:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761986.074848:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789761987.035056:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789761987.035107:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789761987.035127:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761987.035178:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761987.035206:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789761987.035219:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789761987.035247:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761983/real 1789761987] req@ffff93f8517c0380 x1876699565728256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761987.035283:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565728256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761987.035296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761987.035300:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761987.035304:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565728256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761987.035313:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761983/real 1789761987] req@ffff93f8517c1500 x1876699565728384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761987.035326:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565728384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761987.035334:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761987.035338:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761987.035341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565728384:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761987.035348:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761983/real 1789761987] req@ffff93f8517c2680 x1876699565728128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761987.035360:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565728128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761987.035373:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761987.035376:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761987.035380:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565728128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761987.035409:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761983/real 1789761987] req@ffff93f8517c0a80 x1876699565728512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761987.035422:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565728512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761987.035430:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761987.035432:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761987.035436:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565728512:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761987.098799:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761988.122730:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761989.082853:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761989.082866:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761989.082871:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761989.082884:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1924 00000100:00080000:3.0:1789761989.082894:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761989.082903:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761989.082947:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565728640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761989.082978:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761989.082982:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761989.082984:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761989.082989:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1924 00000100:00080000:3.0:1789761989.082993:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761989.082998:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761989.083009:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789761989.083025:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565728768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761989.083038:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761989.083047:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:2.0:1789761989.083048:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565728640:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761989.083050:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761989.083063:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1924 00000100:00080000:3.0:1789761989.083073:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761989.083078:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761989.083085:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789761989.083102:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565728896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761989.083116:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761989.083120:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761989.083123:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761989.083127:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1924 00000100:00080000:3.0:1789761989.083135:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761989.083141:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761989.083146:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789761989.083156:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789761989.083164:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565729024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789761989.083235:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761989/real 1789761989] req@ffff93f844d64e00 x1876699565728640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761989.083260:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565728640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761989.083272:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761989.083278:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761989.083283:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565728640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789761989.083315:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565728768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761989.083346:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761989.083360:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761989/real 1789761989] req@ffff93f844d65c00 x1876699565728768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761989.083380:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565728768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761989.083389:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761989.083393:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761989.083396:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565728768:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789761989.083403:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565728896:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789761989.083426:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761989.083438:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761989/real 1789761989] req@ffff93f844d66a00 x1876699565728896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761989.083451:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565728896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761989.083475:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761989.083479:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761989.083483:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565728896:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789761989.083490:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565729024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789761989.083516:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789761989.083529:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761989/real 1789761989] req@ffff93f844d67100 x1876699565729024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789761989.083548:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565729024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789761989.083558:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789761989.083561:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789761989.083565:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565729024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761989.146765:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761990.170835:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761991.194761:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761992.218790:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761993.242813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789761994.202749:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761994.202759:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789761994.202763:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761994.202771:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1929 00000100:00080000:3.0:1789761994.202780:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761994.202788:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789761994.202833:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565729152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761994.202866:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761994.202871:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761994.202874:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761994.202879:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1929 00000100:00080000:3.0:1789761994.202884:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761994.202889:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789761994.202900:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761994.202912:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565729152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789761994.202918:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565729280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789761994.202932:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761994.202936:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789761994.202938:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761994.202942:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1929 00000100:00080000:3.0:1789761994.202947:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789761994.202952:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789761994.202978:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761994.203025:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761994/real 1789761994] req@ffff93f844d67800 x1876699565729152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761994.203041:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565729152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1789761994.203045:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00080000:0.0:1789761994.203052:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761994.203063:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789761994.203068:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565729408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761994.203070:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565729152:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789761994.203087:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789761994.203091:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789761994.203094:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789761994.203099:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1929 00000100:00080000:3.0:1789761994.203104:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789761994.203112:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565729280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761994.203137:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761994.203148:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761994/real 1789761994] req@ffff93f844d67480 x1876699565729280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761994.203157:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565729280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761994.203163:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761994.203165:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761994.203168:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565729280:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761994.203175:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565729408:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761994.203200:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761994.203212:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761994/real 1789761994] req@ffff93f844d65880 x1876699565729408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761994.203224:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565729408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789761994.203230:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:0.0:1789761994.203233:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761994.203241:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000008:00100000:3.0:1789761994.203243:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789761994.203247:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565729408:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789761994.203262:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565729536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761994.203307:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565729536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761994.203329:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761994.203339:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761994/real 1789761994] req@ffff93f844d66680 x1876699565729536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761994.203347:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565729536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761994.203353:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761994.203355:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761994.203357:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565729536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761994.266829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761995.290788:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761996.314861:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761997.338768:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789761998.362824:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789761999.322756:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761999.322764:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789761999.322768:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761999.322777:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1934 00000100:00080000:1.0:1789761999.322784:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761999.322790:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789761999.322823:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565729664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761999.322846:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761999.322849:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761999.322850:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761999.322854:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1934 00000100:00080000:1.0:1789761999.322856:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761999.322859:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761999.322866:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761999.322876:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565729792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761999.322882:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565729664:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789761999.322888:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761999.322891:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789761999.322892:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761999.322895:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1934 00000100:00080000:1.0:1789761999.322897:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761999.322900:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789761999.322903:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789761999.322911:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565729920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789761999.322918:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789761999.322920:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789761999.322921:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789761999.322924:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1934 00000100:00080000:1.0:1789761999.322933:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789761999.322936:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789761999.322940:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789761999.322941:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789761999.322951:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565730048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789761999.322979:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761999/real 1789761999] req@ffff93f844cc9500 x1876699565729664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761999.322992:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565729664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761999.322999:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761999.323002:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761999.323005:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565729664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789761999.323022:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565729792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761999.323036:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761999.323045:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761999/real 1789761999] req@ffff93f844ccad80 x1876699565729792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761999.323053:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565729792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761999.323063:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761999.323065:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761999.323068:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565729792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789761999.323071:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565729920:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789761999.323082:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761999.323088:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761999/real 1789761999] req@ffff93f844cc8e00 x1876699565729920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761999.323095:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565729920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761999.323100:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761999.323102:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761999.323104:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565729920:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789761999.323107:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565730048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789761999.323117:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789761999.323123:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789761999/real 1789761999] req@ffff93f844cca680 x1876699565730048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789761999.323129:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565730048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789761999.323135:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789761999.323136:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789761999.323139:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565730048:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789761999.386826:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762000.410788:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762001.434775:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762002.458828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762003.482823:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762004.442791:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762004.442802:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762004.442807:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762004.442817:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1939 00000100:00080000:3.0:1789762004.442829:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762004.442836:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762004.442880:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565730176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762004.442908:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762004.442911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762004.442913:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762004.442916:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1939 00000100:00080000:3.0:1789762004.442919:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762004.442924:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762004.442933:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762004.442942:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565730304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762004.442952:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762004.442956:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762004.442961:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789762004.442963:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565730176:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762004.442967:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1939 00000100:00080000:3.0:1789762004.442976:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762004.442981:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762004.442985:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762004.442999:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565730432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762004.443007:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762004.443010:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762004.443012:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762004.443014:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1939 00000100:00080000:3.0:1789762004.443018:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762004.443021:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762004.443025:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762004.443032:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565730560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762004.443044:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762004.443087:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762004/real 1789762004] req@ffff93f844d64700 x1876699565730176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762004.443101:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565730176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762004.443110:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762004.443113:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762004.443117:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565730176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762004.443145:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565730304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762004.443171:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762004.443182:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762004/real 1789762004] req@ffff93f844d67b80 x1876699565730304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762004.443191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565730304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762004.443197:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762004.443199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762004.443201:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565730304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762004.443206:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565730432:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762004.443220:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762004.443227:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762004/real 1789762004] req@ffff93f844d66680 x1876699565730432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762004.443235:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565730432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762004.443240:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762004.443242:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762004.443244:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565730432:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762004.443248:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565730560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762004.443258:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762004.443265:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762004/real 1789762004] req@ffff93f844d65880 x1876699565730560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762004.443272:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565730560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762004.443278:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762004.443282:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762004.443285:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565730560:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762004.506789:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762005.530848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762006.554817:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762007.578836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762008.602790:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762009.562801:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762009.562812:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762009.562816:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762009.562831:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1944 00000100:00080000:1.0:1789762009.562843:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762009.562852:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762009.562896:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565730688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762009.562919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762009.562922:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762009.562923:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762009.562927:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1944 00000100:00080000:1.0:1789762009.562929:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762009.562932:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762009.562940:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762009.562951:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565730816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762009.562961:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565730688:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762009.562961:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762009.562967:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762009.562970:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762009.562975:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1944 00000100:00080000:1.0:1789762009.562977:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762009.562979:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762009.562982:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762009.562991:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565730944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762009.562998:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762009.563001:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762009.563002:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762009.563004:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1944 00000100:00080000:1.0:1789762009.563006:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762009.563009:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762009.563012:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762009.563018:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565731072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762009.563048:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762009.563113:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762009/real 1789762009] req@ffff93f844cc9c00 x1876699565730688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762009.563138:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565730688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762009.563152:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762009.563157:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762009.563162:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565730688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762009.563193:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565730816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762009.563220:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762009.563236:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762009/real 1789762009] req@ffff93f844ccb100 x1876699565730816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762009.563250:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565730816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762009.563259:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762009.563262:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762009.563266:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565730816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762009.563273:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565730944:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762009.563296:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762009.563330:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762009/real 1789762009] req@ffff93f844ccb480 x1876699565730944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762009.563343:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565730944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762009.563351:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762009.563355:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762009.563359:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565730944:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762009.563365:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565731072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762009.563388:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762009.563400:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762009/real 1789762009] req@ffff93f844ccb800 x1876699565731072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762009.563412:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565731072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762009.563421:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762009.563424:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762009.563427:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565731072:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762009.626858:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762010.650830:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762011.674820:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762012.698798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762013.722824:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762014.682802:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762014.682810:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762014.682813:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762014.682822:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1949 00000100:00080000:2.0:1789762014.682829:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762014.682835:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762014.682874:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565731200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762014.682910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762014.682914:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762014.682916:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762014.682921:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1949 00000100:00080000:2.0:1789762014.682926:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762014.682931:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762014.682941:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762014.682958:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565731200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762014.682958:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565731328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762014.682978:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762014.682981:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762014.682982:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762014.682985:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1949 00000100:00080000:2.0:1789762014.682988:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762014.682991:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762014.682995:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762014.683004:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565731456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762014.683011:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762014.683014:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762014.683023:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762014.683027:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1949 00000800:00000100:0.0:1789762014.683033:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762014.683033:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762014.683041:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762014.683045:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762014.683053:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565731584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762014.683094:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762014/real 1789762014] req@ffff93f8517c0a80 x1876699565731200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762014.683109:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565731200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762014.683117:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762014.683121:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762014.683123:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565731200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762014.683142:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565731328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762014.683161:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762014.683170:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762014/real 1789762014] req@ffff93f8517c2680 x1876699565731328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762014.683178:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565731328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762014.683184:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762014.683187:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762014.683189:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565731328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762014.683192:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565731456:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762014.683204:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762014.683210:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762014/real 1789762014] req@ffff93f8517c1500 x1876699565731456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762014.683218:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565731456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762014.683222:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762014.683224:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762014.683226:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565731456:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762014.683230:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565731584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762014.683240:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762014.683246:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762014/real 1789762014] req@ffff93f8517c0380 x1876699565731584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762014.683253:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565731584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762014.683277:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762014.683281:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762014.683284:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565731584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762014.746754:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762015.770851:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762016.794839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762017.818828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762018.842826:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762019.802750:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762019.802761:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762019.802767:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762019.802779:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1954 00000100:00080000:1.0:1789762019.802790:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762019.802797:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762019.802833:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565731712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762019.802859:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762019.802862:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762019.802864:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762019.802869:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1954 00000100:00080000:1.0:1789762019.802872:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762019.802876:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762019.802886:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762019.802898:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565731840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762019.802900:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565731712:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762019.802917:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762019.802920:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762019.802922:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762019.802927:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1954 00000100:00080000:1.0:1789762019.802931:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762019.802936:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762019.802942:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762019.802955:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565731968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762019.802964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762019.802966:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762019.802970:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789762019.802971:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762019.802975:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1954 00000100:00080000:1.0:1789762019.802980:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762019.802983:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762019.802988:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762019.802997:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565732096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762019.803028:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762019/real 1789762019] req@ffff93f844ccaa00 x1876699565731712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762019.803056:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565731712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762019.803067:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762019.803071:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762019.803076:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565731712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762019.803099:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565731840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762019.803121:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762019.803133:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762019/real 1789762019] req@ffff93f844cc8700 x1876699565731840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762019.803146:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565731840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762019.803155:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762019.803158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762019.803162:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565731840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762019.803167:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565731968:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762019.803184:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762019.803194:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762019/real 1789762019] req@ffff93f844ccbb80 x1876699565731968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762019.803206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565731968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762019.803221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762019.803237:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762019.803241:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565731968:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762019.803247:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565732096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762019.803266:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762019.803276:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762019/real 1789762019] req@ffff93f844cc8000 x1876699565732096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762019.803288:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565732096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762019.803297:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762019.803300:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762019.803304:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565732096:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762019.866829:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762020.890806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762021.914747:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762022.938778:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762023.962773:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762024.922782:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762024.922792:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762024.922797:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762024.922807:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1959 00000100:00080000:3.0:1789762024.922815:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762024.922821:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762024.922856:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565732224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762024.922887:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762024.922891:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762024.922894:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762024.922898:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1959 00000100:00080000:3.0:1789762024.922903:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762024.922907:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762024.922917:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762024.922922:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565732224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762024.922934:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565732352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762024.922943:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762024.922945:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762024.922947:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762024.922950:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1959 00000100:00080000:3.0:1789762024.922953:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762024.922956:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762024.922960:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762024.922970:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565732480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762024.922976:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762024.922979:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:2.0:1789762024.922983:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762024.922983:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762024.922988:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1959 00000100:00080000:3.0:1789762024.922992:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762024.922995:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762024.922999:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762024.923008:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565732608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762024.923033:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762024/real 1789762024] req@ffff93f844d67480 x1876699565732224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762024.923046:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565732224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762024.923055:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762024.923059:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762024.923063:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565732224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762024.923083:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565732352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762024.923097:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762024.923106:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762024/real 1789762024] req@ffff93f844d67800 x1876699565732352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762024.923114:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565732352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762024.923120:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762024.923122:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762024.923124:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565732352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762024.923128:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565732480:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762024.923138:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762024.923143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762024/real 1789762024] req@ffff93f844d67100 x1876699565732480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762024.923151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565732480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762024.923156:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762024.923158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762024.923159:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565732480:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762024.923163:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565732608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762024.923173:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762024.923183:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762024/real 1789762024] req@ffff93f844d66a00 x1876699565732608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762024.923209:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565732608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762024.923218:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762024.923221:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762024.923225:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565732608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762024.986820:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762026.010764:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762027.034784:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762028.058845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762029.082830:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762030.042809:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762030.042828:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762030.042836:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762030.042853:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1965 00000100:00080000:3.0:1789762030.042871:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762030.042884:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762030.042942:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565732736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762030.042982:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762030.042987:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762030.042989:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762030.042996:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1965 00000100:00080000:3.0:1789762030.043001:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762030.043007:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762030.043019:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762030.043038:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565732864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762030.043068:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762030.043073:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762030.043076:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762030.043080:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1965 00000100:00080000:3.0:1789762030.043096:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762030.043103:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762030.043111:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762030.043143:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565732992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762030.043175:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789762030.043178:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565732736:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762030.043181:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762030.043186:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762030.043196:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1965 00000100:00080000:3.0:1789762030.043201:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762030.043206:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762030.043213:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762030.043235:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565733120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762030.043316:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762030.043382:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762030/real 1789762030] req@ffff93f844d65c00 x1876699565732736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762030.043414:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565732736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762030.043429:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762030.043435:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762030.043441:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565732736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762030.043450:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565732864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762030.043494:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762030.043517:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762030/real 1789762030] req@ffff93f844d64e00 x1876699565732864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762030.043533:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565732864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762030.043562:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762030.043567:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762030.043572:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565732864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762030.043610:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565732992:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762030.043633:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762030.043660:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762030/real 1789762030] req@ffff93f844d66d80 x1876699565732992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762030.043672:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565732992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762030.043679:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762030.043683:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762030.043686:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565732992:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762030.043708:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565733120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762030.043738:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762030.043754:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762030/real 1789762030] req@ffff93f844d64000 x1876699565733120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762030.043771:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565733120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762030.043783:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762030.043787:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762030.043793:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565733120:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762030.106821:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762031.130813:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762032.154834:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762033.178844:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762034.202822:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762035.162773:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762035.162781:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762035.162785:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762035.162793:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1970 00000100:00080000:1.0:1789762035.162803:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762035.162809:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762035.162851:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565733248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762035.162876:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762035.162879:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762035.162880:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762035.162894:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1970 00000100:00080000:1.0:1789762035.162897:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762035.162899:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762035.162909:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762035.162921:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565733376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762035.162931:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789762035.162935:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565733248:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762035.162936:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762035.162943:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762035.162959:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1970 00000100:00080000:1.0:1789762035.162961:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762035.162963:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762035.162966:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762035.162977:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565733504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762035.162985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762035.162987:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762035.162989:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762035.162992:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1970 00000100:00080000:1.0:1789762035.162994:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762035.162997:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762035.163001:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762035.163008:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565733632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762035.163031:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762035.163104:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762035/real 1789762035] req@ffff93f844cc9880 x1876699565733248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762035.163138:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565733248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762035.163149:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762035.163164:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762035.163168:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565733248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762035.163194:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565733376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762035.163213:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762035.163223:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762035/real 1789762035] req@ffff93f844cc8380 x1876699565733376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762035.163232:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565733376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762035.163237:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762035.163240:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762035.163242:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565733376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762035.163246:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565733504:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762035.163259:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762035.163266:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762035/real 1789762035] req@ffff93f844cc9f80 x1876699565733504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762035.163273:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565733504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762035.163279:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762035.163280:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762035.163282:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565733504:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762035.163286:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565733632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762035.163297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762035.163303:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762035/real 1789762035] req@ffff93f844cc9180 x1876699565733632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762035.163311:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565733632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762035.163316:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762035.163318:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762035.163320:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565733632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762035.226810:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762036.250824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762037.274821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762038.298958:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762039.322847:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762040.282760:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762040.282772:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762040.282777:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762040.282791:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1975 00000100:00080000:1.0:1789762040.282804:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762040.282813:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762040.282859:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565733760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762040.282889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762040.282892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762040.282894:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762040.282899:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1975 00000100:00080000:1.0:1789762040.282903:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762040.282907:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762040.282917:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762040.282925:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565733760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762040.282935:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565733888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762040.282946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762040.282950:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762040.282952:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762040.282956:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1975 00000100:00080000:1.0:1789762040.282959:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762040.282963:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762040.282969:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762040.282980:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565734016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762040.282991:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762040.282995:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762040.282998:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789762040.282998:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762040.283006:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1975 00000100:00080000:1.0:1789762040.283011:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762040.283016:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762040.283020:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762040.283032:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565734144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762040.283048:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762040/real 1789762040] req@ffff93f844cc8a80 x1876699565733760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762040.283065:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565733760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762040.283088:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762040.283091:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762040.283095:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565733760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762040.283123:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565733888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762040.283143:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762040.283153:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762040/real 1789762040] req@ffff93f844cca300 x1876699565733888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762040.283162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565733888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762040.283168:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762040.283170:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762040.283172:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565733888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762040.283178:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565734016:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762040.283192:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762040.283199:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762040/real 1789762040] req@ffff93f844cc9180 x1876699565734016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762040.283207:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565734016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762040.283213:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762040.283215:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762040.283216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565734016:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762040.283220:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565734144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762040.283231:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762040.283237:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762040/real 1789762040] req@ffff93f844cc9f80 x1876699565734144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762040.283244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565734144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762040.283250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762040.283252:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762040.283254:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565734144:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762040.346825:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762041.370850:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762042.394838:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762043.418792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762044.442825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762045.402829:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762045.402839:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762045.402843:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762045.402882:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1980 00000100:00080000:1.0:1789762045.402896:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762045.402904:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762045.402977:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565734272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762045.403025:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762045.403028:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762045.403029:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762045.403044:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1980 00000100:00080000:1.0:1789762045.403047:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762045.403051:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762045.403092:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762045.403114:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565734400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762045.403118:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565734272:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762045.403137:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762045.403142:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762045.403144:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762045.403150:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1980 00000100:00080000:1.0:1789762045.403155:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762045.403160:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762045.403166:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762045.403180:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565734528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762045.403192:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762045.403211:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762045.403213:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789762045.403220:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762045.403221:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1980 00000100:00080000:1.0:1789762045.403229:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762045.403234:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762045.403240:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762045.403253:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565734656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762045.403300:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762045/real 1789762045] req@ffff93f844cc8380 x1876699565734272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762045.403319:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565734272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762045.403328:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762045.403332:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762045.403335:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565734272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762045.403359:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565734400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762045.403377:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762045.403386:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762045/real 1789762045] req@ffff93f844cc9880 x1876699565734400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762045.403395:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565734400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762045.403401:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762045.403404:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762045.403406:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565734400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762045.403409:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565734528:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762045.403421:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762045.403427:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762045/real 1789762045] req@ffff93f844cc8000 x1876699565734528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762045.403435:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565734528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762045.403440:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762045.403442:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762045.403443:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565734528:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762045.403446:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565734656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762045.403457:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762045.403462:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762045/real 1789762045] req@ffff93f844ccbb80 x1876699565734656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762045.403476:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565734656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762045.403487:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762045.403491:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762045.403495:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565734656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762045.466857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762046.490837:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762047.514792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762048.538772:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762049.562778:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762050.522718:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762050.522725:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762050.522728:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762050.522737:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1985 00000100:00080000:1.0:1789762050.522745:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762050.522751:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762050.522791:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565734784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762050.522812:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762050.522815:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762050.522816:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762050.522819:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1985 00000100:00080000:1.0:1789762050.522822:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762050.522824:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762050.522833:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762050.522844:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565734784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762050.522845:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565734912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762050.522859:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762050.522861:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762050.522862:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762050.522865:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1985 00000100:00080000:1.0:1789762050.522867:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762050.522870:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762050.522873:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762050.522882:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565735040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762050.522889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762050.522891:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762050.522892:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762050.522895:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1985 00000100:00080000:1.0:1789762050.522901:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762050.522904:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762050.522908:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762050.522916:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565735168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762050.522952:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565734912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762050.522978:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565735040:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762050.523010:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565735168:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762050.586798:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762051.610832:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762052.634801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789762053.594941:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789762053.595003:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789762053.595020:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762053.595057:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762053.595077:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762053.595088:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789762053.595101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762050/real 1789762053] req@ffff93f844ccaa00 x1876699565734912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762053.595139:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565734912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762053.595155:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762053.595162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762053.595167:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565734912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762053.595177:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762050/real 1789762053] req@ffff93f844ccb800 x1876699565735040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762053.595190:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565735040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762053.595199:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762053.595204:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762053.595208:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565735040:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762053.595217:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762050/real 1789762053] req@ffff93f844ccb480 x1876699565735168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762053.595229:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565735168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762053.595238:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762053.595241:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762053.595245:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565735168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762053.595253:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762050/real 1789762053] req@ffff93f844cc8700 x1876699565734784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762053.595266:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565734784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762053.595273:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762053.595277:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762053.595280:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565734784:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789762053.658823:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762054.682918:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762055.642756:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762055.642764:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762055.642768:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762055.642777:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1990 00000100:00080000:1.0:1789762055.642786:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762055.642792:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762055.642823:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565735296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762055.642844:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762055.642846:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762055.642848:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762055.642851:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1990 00000100:00080000:1.0:1789762055.642853:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762055.642856:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762055.642864:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762055.642873:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565735424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762055.642885:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789762055.642887:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565735296:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762055.642891:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762055.642897:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762055.642901:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1990 00000100:00080000:1.0:1789762055.642904:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762055.642906:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762055.642910:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762055.642922:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565735552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762055.642929:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762055.642932:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762055.642933:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762055.642936:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1990 00000100:00080000:1.0:1789762055.642939:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762055.642945:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762055.642949:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762055.642971:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565735680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762055.643000:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762055.643079:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762055/real 1789762055] req@ffff93f844ccb100 x1876699565735296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762055.643097:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565735296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762055.643109:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762055.643113:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762055.643118:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565735296:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762055.643152:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565735424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762055.643188:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762055.643204:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762055/real 1789762055] req@ffff93f844cc9c00 x1876699565735424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762055.643219:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565735424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762055.643229:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762055.643233:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762055.643242:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565735424:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762055.643251:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565735552:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762055.643276:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762055.643289:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762055/real 1789762055] req@ffff93f844cca680 x1876699565735552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762055.643302:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565735552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762055.643311:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762055.643315:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762055.643318:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565735552:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762055.643326:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565735680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762055.643349:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762055.643362:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762055/real 1789762055] req@ffff93f844cc8e00 x1876699565735680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762055.643375:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565735680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762055.643388:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762055.643392:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762055.643395:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565735680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762055.706783:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762056.730842:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762057.754862:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762058.778814:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762059.802856:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762060.762795:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762060.762806:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762060.762811:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762060.762822:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 1995 00000100:00080000:3.0:1789762060.762841:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762060.762849:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762060.762898:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565735808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762060.762944:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762060.762949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762060.762950:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762060.762954:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1995 00000100:00080000:3.0:1789762060.762958:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762060.762961:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762060.762971:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762060.762983:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565735936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762060.762992:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762060.762995:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762060.762997:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789762060.763000:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565735808:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762060.763002:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 1995 00000100:00080000:3.0:1789762060.763008:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762060.763012:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762060.763015:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762060.763023:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565736064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762060.763035:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762060.763038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762060.763040:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762060.763042:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 1995 00000100:00080000:3.0:1789762060.763044:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762060.763047:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762060.763050:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762060.763056:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565736192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762060.763091:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762060.763149:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762060/real 1789762060] req@ffff93f844d65f80 x1876699565735808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762060.763166:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565735808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762060.763175:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762060.763178:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762060.763181:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565735808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762060.763208:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565735936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762060.763225:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762060.763234:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762060/real 1789762060] req@ffff93f844d64a80 x1876699565735936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762060.763243:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565735936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762060.763249:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762060.763251:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762060.763254:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565735936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762060.763257:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565736064:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762060.763268:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762060.763274:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762060/real 1789762060] req@ffff93f844d64380 x1876699565736064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762060.763282:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565736064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762060.763287:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762060.763289:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762060.763291:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565736064:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762060.763294:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565736192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762060.763304:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762060.763310:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762060/real 1789762060] req@ffff93f844d65500 x1876699565736192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762060.763317:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565736192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762060.763327:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762060.763330:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762060.763332:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565736192:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762060.826821:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762061.850862:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762062.874777:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762063.898859:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762064.922775:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762065.882777:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762065.882795:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762065.882801:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762065.882816:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2000 00000100:00080000:3.0:1789762065.882829:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762065.882839:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762065.882896:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565736320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762065.882940:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762065.882946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762065.882950:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762065.882959:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2000 00000100:00080000:3.0:1789762065.882966:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762065.882973:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762065.882986:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762065.882991:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565736320:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762065.883010:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565736448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762065.883028:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762065.883032:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762065.883036:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789762065.883061:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762065.883105:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762065/real 1789762065] req@ffff93f844d66300 x1876699565736320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762065.883118:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2000 00000100:00100000:2.0:1789762065.883125:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565736320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762065.883127:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762065.883145:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:2.0:1789762065.883146:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762065.883160:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762065.883164:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565736320:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789762065.883164:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762065.883203:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565736448:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762065.883209:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565736576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762065.883224:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762065.883231:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762065.883238:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:2.0:1789762065.883242:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762065/real 1789762065] req@ffff93f844d65180 x1876699565736448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762065.883243:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789762065.883265:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565736448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762065.883272:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762065.883275:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762065.883277:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565736448:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762065.883287:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565736576:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762065.883301:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762065.883311:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762065/real 1789762065] req@ffff93f844d65500 x1876699565736576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762065.883324:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565736576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762065.883333:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762065.883338:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762065.883341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565736576:192.168.203.147@tcp:8:kworker.0 00000100:00080000:3.0:1789762065.883400:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2000 00000100:00080000:3.0:1789762065.883409:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762065.883416:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762065.883469:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762065.883489:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565736704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762065.883541:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565736704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762065.883564:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762065.883574:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762065/real 1789762065] req@ffff93f844d64380 x1876699565736704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762065.883583:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565736704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762065.883594:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762065.883596:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762065.883598:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565736704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762065.946830:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762066.970775:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762067.994776:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762069.018785:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762070.042820:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762071.002738:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762071.002749:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762071.002755:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762071.002766:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2005 00000100:00080000:3.0:1789762071.002775:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762071.002783:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762071.002823:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565736832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762071.002866:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762071.002871:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762071.002873:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762071.002877:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2005 00000100:00080000:3.0:1789762071.002882:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762071.002888:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762071.002898:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762071.002910:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565736832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762071.002915:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565736960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762071.002926:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762071.002929:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762071.002932:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762071.002936:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2005 00000100:00080000:3.0:1789762071.002940:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762071.002945:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762071.002951:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762071.002965:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565737088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762071.002976:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1789762071.002983:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762071.002984:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762071.002989:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762071.002995:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2005 00000100:00080000:3.0:1789762071.003000:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762071.003005:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762071.003010:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762071.003023:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565737216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762071.003045:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762071/real 1789762071] req@ffff93f844d64a80 x1876699565736832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762071.003069:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565736832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762071.003088:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762071.003093:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762071.003097:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565736832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762071.003124:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565736960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762071.003155:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762071.003171:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762071/real 1789762071] req@ffff93f844d65f80 x1876699565736960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762071.003185:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565736960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762071.003194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762071.003198:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762071.003202:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565736960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762071.003209:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565737088:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762071.003231:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762071.003243:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762071/real 1789762071] req@ffff93f844d64000 x1876699565737088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762071.003255:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565737088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762071.003271:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762071.003274:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762071.003278:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565737088:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762071.003282:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565737216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762071.003297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762071.003305:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762071/real 1789762071] req@ffff93f844d66d80 x1876699565737216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762071.003312:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565737216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762071.003318:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762071.003320:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762071.003322:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565737216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762071.066772:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762072.090803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762073.114766:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762074.138789:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762075.162756:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762076.122834:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762076.122848:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762076.122854:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762076.122867:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2011 00000100:00080000:1.0:1789762076.122881:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762076.122891:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762076.122935:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565737344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762076.122964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762076.122968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762076.122969:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762076.122972:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2011 00000100:00080000:1.0:1789762076.122976:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762076.122979:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762076.122988:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762076.122999:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565737472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762076.123010:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565737344:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762076.123015:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762076.123025:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762076.123027:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762076.123031:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2011 00000100:00080000:1.0:1789762076.123034:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762076.123038:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762076.123042:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762076.123053:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565737600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762076.123062:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762076.123065:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762076.123066:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762076.123069:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2011 00000100:00080000:1.0:1789762076.123076:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762076.123079:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762076.123083:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789762076.123088:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762076.123097:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565737728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762076.123135:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762076/real 1789762076] req@ffff93f844ccad80 x1876699565737344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762076.123150:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565737344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762076.123159:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762076.123162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762076.123165:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565737344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762076.123188:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565737472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762076.123215:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762076.123230:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762076/real 1789762076] req@ffff93f844cc9500 x1876699565737472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762076.123244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565737472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762076.123251:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762076.123253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762076.123255:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565737472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762076.123260:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565737600:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762076.123278:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762076.123286:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762076/real 1789762076] req@ffff93f844cc8e00 x1876699565737600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762076.123294:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565737600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762076.123300:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762076.123302:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762076.123304:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565737600:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762076.123308:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565737728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762076.123319:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762076.123325:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762076/real 1789762076] req@ffff93f844cca680 x1876699565737728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762076.123332:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565737728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762076.123337:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762076.123339:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762076.123341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565737728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762076.186785:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762077.210791:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762078.234758:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762079.258855:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762080.282785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762081.242799:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762081.242822:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762081.242827:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762081.242838:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2016 00000100:00080000:3.0:1789762081.242847:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762081.242861:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762081.242895:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565737856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762081.242913:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762081.242916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762081.242917:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762081.242920:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2016 00000100:00080000:3.0:1789762081.242922:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762081.242926:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762081.242934:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762081.242943:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565737856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762081.242945:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565737984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762081.242958:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762081.242960:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762081.242961:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762081.242964:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2016 00000100:00080000:3.0:1789762081.242966:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762081.242968:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762081.242972:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762081.242983:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565738112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762081.242995:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762081.242999:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762081.243000:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789762081.243006:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762081.243007:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2016 00000100:00080000:3.0:1789762081.243014:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762081.243020:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762081.243026:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762081.243040:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565738240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762081.243055:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762081/real 1789762081] req@ffff93f844d64e00 x1876699565737856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762081.243076:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565737856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762081.243088:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762081.243103:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762081.243106:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565737856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762081.243128:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565737984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762081.243146:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762081.243166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762081/real 1789762081] req@ffff93f844d65c00 x1876699565737984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762081.243180:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565737984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762081.243196:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762081.243199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762081.243201:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565737984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762081.243206:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565738112:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762081.243222:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762081.243230:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762081/real 1789762081] req@ffff93f844d66a00 x1876699565738112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762081.243238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565738112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762081.243244:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762081.243246:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762081.243248:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565738112:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762081.243251:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565738240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762081.243261:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762081.243267:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762081/real 1789762081] req@ffff93f844d67100 x1876699565738240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762081.243284:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565738240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762081.243289:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762081.243291:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762081.243293:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565738240:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762081.306852:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762082.330779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762083.354884:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762084.378920:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762085.402839:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762086.362746:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762086.362754:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762086.362758:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762086.362765:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2021 00000100:00080000:3.0:1789762086.362773:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762086.362779:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762086.362812:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565738368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762086.362838:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762086.362841:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762086.362842:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762086.362845:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2021 00000100:00080000:3.0:1789762086.362848:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762086.362850:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762086.362859:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762086.362868:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565738368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762086.362871:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565738496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762086.362881:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762086.362883:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762086.362884:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762086.362887:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2021 00000100:00080000:3.0:1789762086.362889:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762086.362891:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762086.362894:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762086.362903:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565738624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762086.362910:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762086.362913:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762086.362914:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762086.362917:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2021 00000100:00080000:3.0:1789762086.362920:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762086.362922:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762086.362925:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789762086.362930:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762086.362935:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565738752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762086.362974:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762086/real 1789762086] req@ffff93f844d67800 x1876699565738368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762086.362989:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565738368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762086.362998:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762086.363001:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762086.363004:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565738368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762086.363024:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565738496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762086.363046:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762086.363056:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762086/real 1789762086] req@ffff93f844d67480 x1876699565738496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762086.363064:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565738496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762086.363069:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762086.363072:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762086.363074:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565738496:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762086.363078:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565738624:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762086.363088:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762086.363095:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762086/real 1789762086] req@ffff93f844d65880 x1876699565738624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762086.363103:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565738624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762086.363108:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762086.363110:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762086.363112:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565738624:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762086.363115:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565738752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762086.363133:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762086.363144:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762086/real 1789762086] req@ffff93f844d66680 x1876699565738752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762086.363157:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565738752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762086.363166:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762086.363170:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762086.363174:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565738752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762086.426787:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762087.450818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762088.474871:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762089.498869:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762090.522824:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762091.482813:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762091.482824:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762091.482828:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762091.482839:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2026 00000100:00080000:1.0:1789762091.482849:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762091.482856:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762091.482900:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565738880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762091.482925:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762091.482928:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762091.482930:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762091.482933:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2026 00000100:00080000:1.0:1789762091.482936:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762091.482939:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762091.482948:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762091.482957:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565739008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762091.482963:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565738880:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762091.482969:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762091.482973:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762091.482975:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762091.482978:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2026 00000100:00080000:1.0:1789762091.482980:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762091.482982:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762091.482986:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762091.482994:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565739136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762091.483001:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762091.483003:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762091.483004:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762091.483007:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2026 00000100:00080000:1.0:1789762091.483009:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762091.483012:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762091.483016:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762091.483023:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565739264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762091.483039:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762091.483098:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762091/real 1789762091] req@ffff93f844cc9c00 x1876699565738880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762091.483116:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565738880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762091.483125:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762091.483127:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762091.483130:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565738880:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762091.483156:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565739008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762091.483173:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762091.483182:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762091/real 1789762091] req@ffff93f844ccb100 x1876699565739008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762091.483191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565739008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762091.483197:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762091.483199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762091.483201:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565739008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762091.483205:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565739136:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762091.483216:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762091.483222:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762091/real 1789762091] req@ffff93f844cc8700 x1876699565739136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762091.483230:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565739136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762091.483235:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762091.483237:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762091.483239:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565739136:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762091.483242:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565739264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762091.483253:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762091.483259:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762091/real 1789762091] req@ffff93f844ccb480 x1876699565739264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762091.483267:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565739264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762091.483272:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762091.483273:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762091.483275:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565739264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762091.546872:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762092.570818:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762093.594780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762094.618778:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762095.642823:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762096.603127:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762096.603138:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762096.603144:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762096.603156:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2031 00000100:00080000:2.0:1789762096.603167:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762096.603177:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762096.603221:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565739392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762096.603245:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762096.603250:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762096.603252:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762096.603257:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2031 00000100:00080000:2.0:1789762096.603262:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762096.603269:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762096.603287:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762096.603302:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565739392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762096.603313:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565739520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762096.603332:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762096.603336:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762096.603338:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762096.603343:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2031 00000100:00080000:2.0:1789762096.603347:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762096.603352:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762096.603358:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762096.603384:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565739648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762096.603394:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762096.603397:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762096.603401:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762096.603405:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2031 00000100:00080000:2.0:1789762096.603410:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762096.603414:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789762096.603422:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1789762096.603423:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762096.603451:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565739776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762096.603509:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762096/real 1789762096] req@ffff93f8517c1180 x1876699565739392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762096.603538:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565739392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762096.603553:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762096.603560:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762096.603569:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565739392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762096.603607:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565739520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762096.603637:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762096.603700:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762096/real 1789762096] req@ffff93f8517c3100 x1876699565739520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762096.603710:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565739520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762096.603718:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762096.603722:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762096.603726:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565739520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762096.603735:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565739648:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762096.603760:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762096.603774:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762096/real 1789762096] req@ffff93f8517c2a00 x1876699565739648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762096.603787:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565739648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762096.603793:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762096.603796:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762096.603798:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565739648:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762096.603803:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565739776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762096.603820:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762096.603828:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762096/real 1789762096] req@ffff93f8517c2d80 x1876699565739776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762096.603836:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565739776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762096.603842:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762096.603843:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762096.603845:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565739776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762096.666840:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762097.690839:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762098.714792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762099.738801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762100.762883:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762101.722774:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762101.722787:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762101.722793:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762101.722805:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2036 00000100:00080000:1.0:1789762101.722820:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762101.722840:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762101.722901:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565739904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762101.722942:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762101.722948:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762101.722950:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762101.722957:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2036 00000100:00080000:1.0:1789762101.722962:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762101.722979:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762101.722992:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565739904:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762101.722993:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762101.723022:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565740032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762101.723040:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762101.723044:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762101.723046:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762101.723051:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2036 00000100:00080000:1.0:1789762101.723068:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762101.723073:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762101.723079:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789762101.723081:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762101.723097:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565740160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762101.723120:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762101.723123:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762101.723125:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762101.723129:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2036 00000100:00080000:1.0:1789762101.723134:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762101.723138:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762101.723143:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762101.723149:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762101/real 1789762101] req@ffff93f844ccb800 x1876699565739904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762101.723156:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565740288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762101.723194:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565739904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762101.723208:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762101.723212:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762101.723218:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565739904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762101.723268:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565740032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762101.723304:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762101.723321:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762101/real 1789762101] req@ffff93f844ccaa00 x1876699565740032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762101.723339:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565740032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762101.723351:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762101.723354:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762101.723374:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565740032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762101.723381:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565740160:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762101.723406:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762101.723425:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762101/real 1789762101] req@ffff93f844ccbb80 x1876699565740160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762101.723438:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565740160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762101.723464:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762101.723468:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762101.723473:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565740160:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762101.723481:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565740288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762101.723521:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762101.723545:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762101/real 1789762101] req@ffff93f844cc8000 x1876699565740288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762101.723567:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565740288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762101.723576:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762101.723579:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762101.723582:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565740288:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762101.786873:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762102.810835:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762103.834827:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762104.858849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762105.882775:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762106.842777:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762106.842789:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762106.842794:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762106.842806:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2041 00000100:00080000:1.0:1789762106.842818:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762106.842826:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762106.842870:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565740416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762106.842897:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762106.842900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762106.842902:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762106.842905:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2041 00000100:00080000:1.0:1789762106.842907:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762106.842910:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762106.842918:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762106.842930:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565740544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762106.842938:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762106.842941:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762106.842942:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762106.842945:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2041 00000100:00080000:1.0:1789762106.842947:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762106.842949:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762106.842953:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762106.842956:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565740416:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762106.842963:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565740672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762106.842971:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762106.842973:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762106.842974:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762106.842977:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2041 00000100:00080000:1.0:1789762106.842980:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762106.842982:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762106.842985:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762106.842992:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565740800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762106.843038:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762106.843090:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762106/real 1789762106] req@ffff93f844cc9880 x1876699565740416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762106.843107:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565740416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762106.843116:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762106.843119:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762106.843122:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565740416:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762106.843128:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565740544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762106.843147:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762106.843157:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762106/real 1789762106] req@ffff93f844cc8380 x1876699565740544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762106.843165:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565740544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762106.843171:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762106.843173:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762106.843176:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565740544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762106.843197:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565740672:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762106.843210:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762106.843217:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762106/real 1789762106] req@ffff93f844cc9f80 x1876699565740672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762106.843225:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565740672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762106.843230:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762106.843232:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762106.843234:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565740672:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762106.843237:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565740800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762106.843247:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762106.843254:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762106/real 1789762106] req@ffff93f844cc9180 x1876699565740800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762106.843262:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565740800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762106.843267:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762106.843269:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762106.843271:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565740800:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762106.906840:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762107.930770:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762108.954904:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762109.978764:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762111.002912:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762111.962755:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762111.962766:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762111.962771:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762111.962783:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2046 00000100:00080000:3.0:1789762111.962796:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762111.962805:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762111.962861:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565740928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762111.962886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762111.962889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762111.962891:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762111.962894:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2046 00000100:00080000:3.0:1789762111.962898:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762111.962903:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762111.962927:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762111.962948:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565741056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762111.962949:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565740928:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762111.962972:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762111.962975:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762111.962977:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762111.962980:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2046 00000100:00080000:3.0:1789762111.962985:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762111.962988:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762111.962993:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762111.963011:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565741184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762111.963024:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762111.963028:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762111.963033:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789762111.963035:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762111.963042:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2046 00000100:00080000:3.0:1789762111.963048:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762111.963055:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762111.963061:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762111.963090:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565741312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762111.963127:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762111/real 1789762111] req@ffff93f844d67b80 x1876699565740928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762111.963172:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565740928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762111.963187:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762111.963202:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762111.963223:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565740928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762111.963265:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565741056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762111.963302:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762111.963321:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762111/real 1789762111] req@ffff93f844d64700 x1876699565741056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762111.963340:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565741056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762111.963362:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762111.963367:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762111.963373:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565741056:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762111.963382:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565741184:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762111.963413:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762111.963431:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762111/real 1789762111] req@ffff93f844d66680 x1876699565741184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762111.963450:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565741184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762111.963463:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762111.963487:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762111.963493:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565741184:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762111.963502:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565741312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762111.963552:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762111.963596:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762111/real 1789762111] req@ffff93f844d65880 x1876699565741312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762111.963613:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565741312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762111.963623:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762111.963637:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762111.963642:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565741312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762112.026810:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762113.050820:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762114.074811:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762115.098786:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762116.122890:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762117.082741:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762117.082754:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762117.082762:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762117.082775:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2052 00000100:00080000:1.0:1789762117.082787:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762117.082797:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762117.082848:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565741440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762117.082894:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762117.082899:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762117.082901:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762117.082906:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2052 00000100:00080000:1.0:1789762117.082910:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762117.082915:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762117.082926:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762117.082940:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565741440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762117.082943:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565741568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762117.082957:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762117.082961:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762117.082963:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762117.082967:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2052 00000100:00080000:1.0:1789762117.082971:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762117.082976:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762117.082982:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762117.082996:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565741696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762117.083008:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762117.083012:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762117.083014:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762117.083018:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2052 00000100:00080000:1.0:1789762117.083023:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762117.083027:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762117.083032:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762117.083044:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565741824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762117.083065:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565741568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762117.083096:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565741696:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762117.083116:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565741824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762117.146776:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762118.170801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762119.194763:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789762120.154913:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789762120.154949:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789762120.154963:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762120.155003:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762120.155013:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762120.155021:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789762120.155060:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762117/real 1789762120] req@ffff93f844cc8a80 x1876699565741568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762120.155087:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565741568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762120.155100:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762120.155105:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762120.155110:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565741568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762120.155118:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762117/real 1789762120] req@ffff93f844cc9180 x1876699565741696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762120.155130:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565741696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762120.155139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762120.155142:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762120.155146:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565741696:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762120.155153:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762117/real 1789762120] req@ffff93f844cc9f80 x1876699565741824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762120.155167:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565741824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762120.155174:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762120.155176:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762120.155179:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565741824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762120.155183:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762117/real 1789762120] req@ffff93f844cca300 x1876699565741440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762120.155191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565741440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762120.155196:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762120.155198:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762120.155200:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565741440:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789762120.218777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762121.242765:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762122.202783:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762122.202793:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762122.202797:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762122.202806:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2057 00000100:00080000:1.0:1789762122.202813:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762122.202820:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762122.202865:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565741952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762122.202891:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762122.202895:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762122.202898:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762122.202917:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2057 00000100:00080000:1.0:1789762122.202921:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762122.202926:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762122.202937:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762122.202945:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565741952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762122.202955:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565742080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762122.202986:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762122.202991:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762122.202993:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762122.202998:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2057 00000100:00080000:1.0:1789762122.203029:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762122.203034:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789762122.203042:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789762122.203042:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762122.203078:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565742208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762122.203102:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762122.203105:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762122.203107:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762122.203111:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2057 00000100:00080000:1.0:1789762122.203115:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762122.203121:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762122.203123:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762122/real 1789762122] req@ffff93f844cc8380 x1876699565741952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1789762122.203127:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762122.203146:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565742336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762122.203153:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565741952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762122.203170:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762122.203175:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762122.203180:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565741952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762122.203210:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565742080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762122.203237:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762122.203251:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762122/real 1789762122] req@ffff93f844cc9880 x1876699565742080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762122.203264:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565742080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762122.203272:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762122.203277:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762122.203280:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565742080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762122.203286:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565742208:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762122.203307:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762122.203321:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762122/real 1789762122] req@ffff93f844cc8000 x1876699565742208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762122.203332:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565742208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762122.203339:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762122.203343:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762122.203345:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565742208:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762122.203351:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565742336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762122.203370:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762122.203381:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762122/real 1789762122] req@ffff93f844ccbb80 x1876699565742336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762122.203391:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565742336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762122.203398:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762122.203401:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762122.203404:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565742336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762122.266902:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762123.290790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762124.314843:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762125.338848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762126.362773:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762127.322753:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762127.322766:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762127.322771:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762127.322782:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2062 00000100:00080000:0.0:1789762127.322791:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762127.322799:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762127.322848:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565742464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762127.322874:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762127.322879:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762127.322881:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762127.322886:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2062 00000100:00080000:0.0:1789762127.322890:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762127.322895:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762127.322905:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762127.322912:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565742464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762127.322918:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565742592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762127.322934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762127.322937:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762127.322938:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762127.322942:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2062 00000100:00080000:0.0:1789762127.322946:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762127.322949:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762127.322954:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762127.322962:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565742720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762127.322971:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762127.322975:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762127.322976:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762127.322979:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2062 00000100:00080000:0.0:1789762127.322982:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762127.322986:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1789762127.322988:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1789762127.322991:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762127.323000:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565742848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762127.323040:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762127/real 1789762127] req@ffff93f850d41c00 x1876699565742464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762127.323061:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565742464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762127.323070:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762127.323073:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762127.323076:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565742464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762127.323100:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565742592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762127.323120:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762127.323129:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762127/real 1789762127] req@ffff93f850d40e00 x1876699565742592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762127.323138:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565742592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762127.323143:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762127.323146:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762127.323148:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565742592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762127.323151:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565742720:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762127.323163:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762127.323169:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762127/real 1789762127] req@ffff93f850d40700 x1876699565742720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762127.323181:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565742720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762127.323187:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762127.323189:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762127.323190:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565742720:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762127.323194:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565742848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762127.323215:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762127.323222:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762127/real 1789762127] req@ffff93f850d43b80 x1876699565742848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762127.323228:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565742848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762127.323234:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762127.323236:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762127.323238:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565742848:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762127.386802:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762128.410794:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762129.434785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762130.458811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762131.482859:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762132.442790:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762132.442800:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762132.442804:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762132.442814:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2067 00000100:00080000:1.0:1789762132.442825:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762132.442833:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762132.442874:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565742976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762132.442905:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762132.442911:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762132.442914:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762132.442920:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2067 00000100:00080000:1.0:1789762132.442926:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762132.442931:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762132.442945:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762132.442955:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565742976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762132.442963:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565743104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762132.442977:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762132.442982:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762132.442984:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762132.442988:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2067 00000100:00080000:1.0:1789762132.442992:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762132.442998:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762132.443005:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762132.443020:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565743232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762132.443030:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762132.443048:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762132.443053:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762132.443055:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762132.443060:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2067 00000100:00080000:1.0:1789762132.443065:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762132.443070:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762132.443076:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762132.443082:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762132/real 1789762132] req@ffff93f844ccaa00 x1876699565742976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762132.443090:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565743360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762132.443116:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565742976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762132.443142:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762132.443148:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762132.443154:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565742976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762132.443185:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565743104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762132.443211:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762132.443226:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762132/real 1789762132] req@ffff93f844ccb800 x1876699565743104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762132.443237:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565743104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762132.443245:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762132.443247:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762132.443250:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565743104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762132.443255:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565743232:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762132.443270:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762132.443278:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762132/real 1789762132] req@ffff93f844ccb480 x1876699565743232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762132.443288:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565743232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762132.443301:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762132.443304:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762132.443307:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565743232:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762132.443311:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565743360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762132.443325:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762132.443333:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762132/real 1789762132] req@ffff93f844cc8700 x1876699565743360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762132.443343:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565743360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762132.443350:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762132.443352:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762132.443354:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565743360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762132.506834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762133.530836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762134.554848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762135.578814:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762136.602763:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762137.562721:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762137.562729:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762137.562733:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762137.562741:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2072 00000100:00080000:1.0:1789762137.562750:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762137.562755:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762137.562788:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565743488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762137.562813:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762137.562817:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762137.562818:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762137.562822:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2072 00000100:00080000:1.0:1789762137.562826:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762137.562831:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762137.562841:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762137.562857:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565743616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762137.562867:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789762137.562870:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565743488:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762137.562871:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762137.562872:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762137.562876:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2072 00000100:00080000:1.0:1789762137.562879:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762137.562882:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762137.562887:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762137.562896:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565743744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762137.562905:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762137.562925:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762137.562928:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762137.562932:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2072 00000100:00080000:1.0:1789762137.562938:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762137.562943:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762137.562950:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789762137.562952:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762137.562962:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565743872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762137.563020:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762137/real 1789762137] req@ffff93f844ccb100 x1876699565743488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762137.563036:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565743488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762137.563047:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762137.563051:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762137.563056:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565743488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762137.563079:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565743616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762137.563103:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762137.563116:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762137/real 1789762137] req@ffff93f844cc9c00 x1876699565743616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762137.563127:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565743616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762137.563135:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762137.563139:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762137.563142:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565743616:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762137.563148:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565743744:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762137.563195:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762137.563218:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762137/real 1789762137] req@ffff93f844cca680 x1876699565743744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762137.563240:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565743744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762137.563254:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762137.563258:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762137.563262:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565743744:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762137.563273:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565743872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762137.563317:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762137.563329:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762137/real 1789762137] req@ffff93f844cc8e00 x1876699565743872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762137.563356:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565743872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762137.563366:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762137.563369:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762137.563372:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565743872:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762137.626872:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762138.650847:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762139.674849:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762140.698824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762141.722831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762142.682785:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762142.682794:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762142.682799:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762142.682807:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2077 00000100:00080000:3.0:1789762142.682819:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762142.682825:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762142.682861:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565744000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762142.682883:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762142.682886:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762142.682887:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762142.682891:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2077 00000100:00080000:3.0:1789762142.682893:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762142.682896:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762142.682903:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762142.682912:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565744128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762142.682919:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762142.682922:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762142.682923:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762142.682927:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2077 00000100:00080000:3.0:1789762142.682929:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762142.682932:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762142.682936:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762142.682941:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565744000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762142.682945:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565744256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762142.682956:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762142.682958:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762142.682960:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762142.682962:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2077 00000100:00080000:3.0:1789762142.682963:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762142.682966:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762142.682969:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762142.682978:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565744384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762142.683014:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762142.683060:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762142/real 1789762142] req@ffff93f844d67480 x1876699565744000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762142.683080:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565744000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762142.683092:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762142.683098:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762142.683102:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565744000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762142.683110:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565744128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762142.683134:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762142.683149:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762142/real 1789762142] req@ffff93f844d67800 x1876699565744128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762142.683169:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565744128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762142.683179:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762142.683182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762142.683186:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565744128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762142.683211:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565744256:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762142.683232:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762142.683245:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762142/real 1789762142] req@ffff93f844d67100 x1876699565744256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762142.683257:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565744256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762142.683267:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762142.683271:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762142.683274:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565744256:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762142.683280:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565744384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762142.683300:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762142.683329:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762142/real 1789762142] req@ffff93f844d66a00 x1876699565744384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762142.683353:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565744384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762142.683368:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762142.683371:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762142.683375:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565744384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762142.746832:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762143.770830:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762144.794788:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762145.818808:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762146.842804:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762147.802738:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762147.802752:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762147.802758:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762147.802772:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2082 00000100:00080000:3.0:1789762147.802785:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762147.802796:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762147.802844:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565744512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762147.802876:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762147.802881:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762147.802882:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762147.802888:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2082 00000100:00080000:3.0:1789762147.802893:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762147.802899:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762147.802908:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565744512:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789762147.802910:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762147.802926:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565744640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762147.802938:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762147.802942:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762147.802944:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762147.802950:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2082 00000100:00080000:3.0:1789762147.802955:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762147.802959:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762147.802965:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789762147.802977:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762147.802982:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565744768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762147.802997:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762147.803001:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762147.803003:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762147.803007:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2082 00000100:00080000:3.0:1789762147.803012:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762147.803017:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762147.803023:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762147.803029:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762147/real 1789762147] req@ffff93f844d65c00 x1876699565744512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762147.803038:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565744896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762147.803049:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565744512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762147.803063:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762147.803067:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762147.803071:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565744512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762147.803099:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565744640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762147.803116:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762147.803125:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762147/real 1789762147] req@ffff93f844d64e00 x1876699565744640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762147.803133:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565744640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762147.803139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762147.803142:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762147.803144:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565744640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762147.803148:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565744768:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762147.803160:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762147.803166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762147/real 1789762147] req@ffff93f844d66d80 x1876699565744768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762147.803174:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565744768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762147.803179:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762147.803181:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762147.803183:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565744768:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762147.803186:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565744896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762147.803197:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762147.803202:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762147/real 1789762147] req@ffff93f844d64000 x1876699565744896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762147.803210:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565744896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762147.803215:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762147.803217:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762147.803219:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565744896:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762147.866822:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762148.890809:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762149.915320:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762150.938791:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762151.962802:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762152.922790:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762152.922805:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762152.922817:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762152.922832:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2087 00000100:00080000:1.0:1789762152.922846:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762152.922855:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762152.922910:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565745024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762152.922945:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762152.922949:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762152.922951:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762152.922956:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2087 00000100:00080000:1.0:1789762152.922961:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762152.922967:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762152.922978:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762152.922982:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565745024:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762152.922997:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565745152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762152.923012:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762152.923023:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762152.923025:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762152.923030:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2087 00000100:00080000:1.0:1789762152.923042:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789762152.923050:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762152.923051:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762152.923059:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762152.923078:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565745280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762152.923091:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762152/real 1789762152] req@ffff93f844cc9500 x1876699565745024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762152.923093:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762152.923110:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762152.923117:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762152.923125:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2087 00000100:00100000:0.0:1789762152.923129:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565745024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762152.923133:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762152.923147:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:0.0:1789762152.923148:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762152.923157:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000008:00100000:1.0:1789762152.923159:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762152.923164:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565745024:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762152.923182:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565745408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762152.923192:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565745152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762152.923242:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762152.923264:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762152/real 1789762152] req@ffff93f844ccad80 x1876699565745152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762152.923274:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565745152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762152.923280:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762152.923282:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762152.923285:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565745152:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762152.923289:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565745280:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762152.923306:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762152.923319:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762152/real 1789762152] req@ffff93f844cc8e00 x1876699565745280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762152.923328:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565745280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762152.923334:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762152.923336:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762152.923338:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565745280:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762152.923350:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565745408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762152.923363:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762152.923370:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762152/real 1789762152] req@ffff93f844cca680 x1876699565745408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762152.923377:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565745408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762152.923382:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762152.923384:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762152.923386:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565745408:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762152.986857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762154.010873:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762155.034829:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762156.058812:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762157.082873:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762158.042742:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762158.042752:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762158.042756:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762158.042766:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2093 00000100:00080000:1.0:1789762158.042776:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762158.042785:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762158.042826:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565745536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762158.042851:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762158.042856:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762158.042858:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762158.042863:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2093 00000100:00080000:1.0:1789762158.042868:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762158.042873:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762158.042883:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565745536:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762158.042884:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762158.042903:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565745664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762158.042916:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762158.042920:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762158.042922:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762158.042926:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2093 00000100:00080000:1.0:1789762158.042932:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762158.042936:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789762158.042940:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789762158.042942:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762158.042957:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565745792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762158.042969:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762158.042972:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762158.042975:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789762158.042979:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762158/real 1789762158] req@ffff93f844cc9c00 x1876699565745536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762158.042983:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2093 00000100:00080000:1.0:1789762158.042990:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762158.042996:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762158.043000:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565745536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1789762158.043003:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00080000:0.0:1789762158.043015:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762158.043018:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762158.043023:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565745536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762158.043023:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565745920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762158.043049:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565745664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762158.043067:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762158.043075:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762158/real 1789762158] req@ffff93f844ccb100 x1876699565745664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762158.043083:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565745664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762158.043089:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762158.043091:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762158.043093:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565745664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762158.043097:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565745792:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762158.043107:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762158.043114:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762158/real 1789762158] req@ffff93f844cc8700 x1876699565745792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762158.043121:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565745792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762158.043127:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762158.043129:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762158.043130:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565745792:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762158.043134:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565745920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762158.043144:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762158.043150:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762158/real 1789762158] req@ffff93f844ccb480 x1876699565745920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762158.043157:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565745920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762158.043162:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762158.043164:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762158.043166:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565745920:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762158.106772:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762159.130827:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762160.154801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762161.178819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762162.202835:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762163.163007:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762163.163019:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762163.163023:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762163.163034:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2098 00000100:00080000:3.0:1789762163.163045:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762163.163053:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762163.163091:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565746048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762163.163117:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762163.163122:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762163.163124:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762163.163128:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2098 00000100:00080000:3.0:1789762163.163133:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762163.163137:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762163.163148:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762163.163170:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565746048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762163.163182:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565746176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762163.163193:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762163.163196:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762163.163198:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762163.163201:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2098 00000100:00080000:3.0:1789762163.163204:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762163.163207:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762163.163212:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762163.163221:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565746304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762163.163228:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1789762163.163230:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762163.163232:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762163.163235:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762163.163239:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2098 00000100:00080000:3.0:1789762163.163242:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762163.163246:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762163.163249:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762163.163258:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565746432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762163.163275:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762163/real 1789762163] req@ffff93f844d65f80 x1876699565746048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762163.163288:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565746048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762163.163297:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762163.163300:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762163.163303:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565746048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762163.163323:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565746176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762163.163339:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762163.163348:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762163/real 1789762163] req@ffff93f844d64a80 x1876699565746176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762163.163356:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565746176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762163.163365:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762163.163368:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762163.163371:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565746176:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762163.163377:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565746304:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762163.163398:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762163.163410:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762163/real 1789762163] req@ffff93f844d64380 x1876699565746304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762163.163422:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565746304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762163.163443:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762163.163446:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762163.163449:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565746304:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762163.163453:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565746432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762163.163469:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762163.163477:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762163/real 1789762163] req@ffff93f844d65500 x1876699565746432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762163.163489:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565746432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762163.163495:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762163.163496:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762163.163498:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565746432:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762163.226836:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762164.250879:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762165.274942:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762166.298778:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762167.322871:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762168.282752:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762168.282759:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762168.282763:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762168.282771:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2103 00000100:00080000:0.0:1789762168.282779:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762168.282785:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762168.282837:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565746560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762168.282862:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762168.282865:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762168.282867:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762168.282869:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2103 00000100:00080000:0.0:1789762168.282872:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762168.282874:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762168.282883:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762168.282893:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565746688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762168.282901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789762168.282904:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565746560:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762168.282905:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762168.282908:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762168.282916:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2103 00000100:00080000:0.0:1789762168.282918:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762168.282921:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762168.282924:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762168.282935:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565746816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762168.282959:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762168.282962:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762168.282963:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762168.282966:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2103 00000100:00080000:0.0:1789762168.282969:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762168.282971:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762168.282974:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762168.282981:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565746944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762168.282990:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762168.283051:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762168/real 1789762168] req@ffff93f850d42300 x1876699565746560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762168.283065:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565746560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762168.283085:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762168.283090:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762168.283095:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565746560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762168.283130:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565746688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762168.283170:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762168.283180:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762168/real 1789762168] req@ffff93f850d40380 x1876699565746688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762168.283189:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565746688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762168.283204:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762168.283206:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762168.283208:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565746688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762168.283212:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565746816:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762168.283224:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762168.283230:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762168/real 1789762168] req@ffff93f850d43b80 x1876699565746816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762168.283237:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565746816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762168.283242:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762168.283244:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762168.283246:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565746816:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762168.283250:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565746944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762168.283260:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762168.283266:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762168/real 1789762168] req@ffff93f850d40700 x1876699565746944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762168.283273:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565746944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762168.283278:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762168.283279:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762168.283281:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565746944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762168.346760:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762169.370788:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762170.394806:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762171.418798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762172.442789:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762173.402808:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762173.402818:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762173.402822:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762173.402831:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2108 00000100:00080000:3.0:1789762173.402840:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762173.402846:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762173.402878:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565747072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762173.402900:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762173.402903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762173.402905:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762173.402908:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2108 00000100:00080000:3.0:1789762173.402910:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762173.402924:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762173.402934:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762173.402945:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565747200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762173.402957:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762173.402963:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1789762173.402965:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565747072:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762173.402966:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762173.402983:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2108 00000100:00080000:3.0:1789762173.402986:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762173.402989:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762173.402992:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762173.403001:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565747328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762173.403009:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762173.403011:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762173.403012:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762173.403015:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2108 00000100:00080000:3.0:1789762173.403017:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762173.403020:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762173.403023:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762173.403029:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565747456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762173.403067:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762173.403151:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762173/real 1789762173] req@ffff93f844d65180 x1876699565747072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762173.403171:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565747072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762173.403200:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762173.403204:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762173.403208:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565747072:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762173.403239:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565747200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762173.403260:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762173.403271:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762173/real 1789762173] req@ffff93f844d66300 x1876699565747200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762173.403280:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565747200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762173.403287:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762173.403289:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762173.403292:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565747200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762173.403297:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565747328:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762173.403312:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762173.403324:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762173/real 1789762173] req@ffff93f844d65500 x1876699565747328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762173.403334:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565747328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762173.403345:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762173.403349:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762173.403355:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565747328:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762173.403364:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565747456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762173.403392:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762173.403417:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762173/real 1789762173] req@ffff93f844d64380 x1876699565747456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762173.403431:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565747456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762173.403440:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762173.403443:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762173.403445:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565747456:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762173.466861:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762174.490967:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762175.514887:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762176.538897:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762177.562807:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762178.522754:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762178.522764:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762178.522769:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762178.522780:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2113 00000100:00080000:1.0:1789762178.522789:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762178.522796:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762178.522841:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565747584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762178.522868:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762178.522871:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762178.522873:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762178.522876:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2113 00000100:00080000:1.0:1789762178.522878:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762178.522881:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762178.522890:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762178.522899:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565747712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762178.522907:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762178.522912:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1789762178.522913:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565747584:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762178.522914:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762178.522922:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2113 00000100:00080000:1.0:1789762178.522925:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762178.522927:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762178.522931:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762178.522939:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565747840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762178.522946:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762178.522948:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762178.522950:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762178.522953:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2113 00000100:00080000:1.0:1789762178.522955:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762178.522958:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762178.522961:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762178.522967:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565747968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762178.522988:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762178.523032:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762178/real 1789762178] req@ffff93f844ccb800 x1876699565747584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762178.523068:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565747584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762178.523079:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762178.523085:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762178.523089:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565747584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762178.523118:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565747712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762178.523141:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762178.523155:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762178/real 1789762178] req@ffff93f844ccaa00 x1876699565747712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762178.523168:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565747712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762178.523177:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762178.523180:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762178.523183:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565747712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762178.523189:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565747840:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762178.523211:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762178.523222:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762178/real 1789762178] req@ffff93f844ccbb80 x1876699565747840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762178.523234:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565747840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762178.523242:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762178.523245:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762178.523248:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565747840:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762178.523254:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565747968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762178.523275:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762178.523288:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762178/real 1789762178] req@ffff93f844cc8000 x1876699565747968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762178.523300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565747968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762178.523308:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762178.523312:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762178.523315:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565747968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762178.586802:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762179.610782:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762180.634761:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762181.658853:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762182.682890:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762183.642749:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762183.642763:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762183.642770:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762183.642783:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2118 00000100:00080000:1.0:1789762183.642797:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762183.642807:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762183.642859:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565748096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762183.642888:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762183.642892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762183.642894:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762183.642898:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2118 00000100:00080000:1.0:1789762183.642902:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762183.642907:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762183.642919:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762183.642933:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565748224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762183.642943:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762183.642947:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762183.642949:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789762183.642956:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565748096:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762183.642958:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2118 00000100:00080000:1.0:1789762183.642966:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762183.642972:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762183.642976:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762183.642987:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565748352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762183.643011:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762183.643018:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762183.643021:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762183.643026:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2118 00000100:00080000:1.0:1789762183.643032:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762183.643036:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762183.643043:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762183.643061:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565748480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762183.643135:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565748224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762183.643187:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565748352:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762183.643212:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565748480:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762183.706791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762184.730829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762185.754809:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1789762186.714951:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1789762186.715018:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1789762186.715038:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789762186.715089:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789762186.715105:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789762186.715115:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789762186.715157:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762183/real 1789762186] req@ffff93f844cc8380 x1876699565748224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762186.715189:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565748224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762186.715204:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762186.715211:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762186.715217:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565748224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762186.715228:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762183/real 1789762186] req@ffff93f844cca300 x1876699565748352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762186.715242:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565748352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762186.715251:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762186.715256:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762186.715261:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565748352:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762186.715282:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762183/real 1789762186] req@ffff93f844cc9f80 x1876699565748480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762186.715296:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565748480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762186.715305:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762186.715309:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762186.715313:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565748480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762186.715320:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762183/real 1789762186] req@ffff93f844cc9880 x1876699565748096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762186.715334:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565748096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762186.715342:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762186.715346:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762186.715349:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565748096:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789762186.778822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762187.802766:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762188.762867:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762188.762881:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762188.762887:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762188.762904:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2123 00000100:00080000:2.0:1789762188.762919:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762188.762930:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762188.763024:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565748608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762188.763053:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762188.763056:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762188.763058:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762188.763078:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2123 00000100:00080000:2.0:1789762188.763083:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762188.763089:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762188.763103:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762188.763123:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565748736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762188.763139:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762188.763143:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762188.763146:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762188.763155:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2123 00000100:00100000:0.0:1789762188.763157:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565748608:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762188.763160:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762188.763171:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762188.763185:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762188.763196:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565748864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762188.763220:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762188.763223:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762188.763224:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762188.763227:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2123 00000100:00080000:2.0:1789762188.763230:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762188.763233:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762188.763237:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762188.763245:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565748992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762188.763283:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762188.763356:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762188/real 1789762188] req@ffff93f8517c0e00 x1876699565748608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762188.763378:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565748608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762188.763389:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762188.763394:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762188.763399:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565748608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762188.763432:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565748736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762188.763459:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762188.763474:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762188/real 1789762188] req@ffff93f8517c3480 x1876699565748736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762188.763486:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565748736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762188.763495:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762188.763499:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762188.763502:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565748736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762188.763509:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565748864:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762188.763531:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762188.763545:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762188/real 1789762188] req@ffff93f8517c3800 x1876699565748864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762188.763556:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565748864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762188.763564:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762188.763567:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762188.763570:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565748864:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762188.763576:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565748992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762188.763595:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762188.763605:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762188/real 1789762188] req@ffff93f8517c1880 x1876699565748992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762188.763616:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565748992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762188.763623:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762188.763626:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762188.763629:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565748992:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762188.826893:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762189.850770:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762190.874811:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762191.898836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762192.922766:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762193.882779:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762193.882792:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762193.882798:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762193.882811:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2128 00000100:00080000:2.0:1789762193.882824:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762193.882834:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762193.882887:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565749120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762193.882917:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762193.882934:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762193.882936:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762193.882941:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2128 00000100:00080000:2.0:1789762193.882946:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762193.882951:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762193.882961:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762193.882977:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565749248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762193.882990:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565749120:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762193.882993:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762193.882999:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762193.883001:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762193.883007:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2128 00000100:00080000:2.0:1789762193.883011:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762193.883015:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762193.883021:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762193.883036:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565749376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762193.883049:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762193.883053:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762193.883055:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762193.883059:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2128 00000100:00080000:2.0:1789762193.883064:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789762193.883065:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762193.883073:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762193.883080:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762193.883093:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565749504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762193.883119:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762193/real 1789762193] req@ffff93f8517c1880 x1876699565749120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762193.883142:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565749120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762193.883157:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762193.883161:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762193.883165:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565749120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762193.883188:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565749248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762193.883205:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762193.883215:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762193/real 1789762193] req@ffff93f8517c3800 x1876699565749248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762193.883226:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565749248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762193.883235:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762193.883239:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762193.883242:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565749248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762193.883249:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565749376:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762193.883271:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762193.883283:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762193/real 1789762193] req@ffff93f8517c3480 x1876699565749376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762193.883291:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565749376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762193.883301:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762193.883304:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762193.883306:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565749376:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762193.883310:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565749504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762193.883323:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762193.883330:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762193/real 1789762193] req@ffff93f8517c0e00 x1876699565749504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762193.883337:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565749504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762193.883343:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762193.883344:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762193.883346:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565749504:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762193.946806:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762194.970792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762195.994844:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762197.018781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762198.042839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762199.002828:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762199.002839:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762199.002843:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762199.002853:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2133 00000100:00080000:3.0:1789762199.002864:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762199.002871:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762199.002925:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565749632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762199.002953:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762199.002958:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762199.002960:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762199.002965:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2133 00000100:00080000:3.0:1789762199.002969:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762199.002974:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762199.002985:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762199.003001:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565749632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762199.003004:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565749760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762199.003017:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762199.003020:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762199.003021:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762199.003024:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2133 00000100:00080000:3.0:1789762199.003028:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762199.003031:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762199.003035:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762199.003046:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565749888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762199.003053:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762199.003056:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762199.003057:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762199.003059:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2133 00000100:00080000:3.0:1789762199.003062:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762199.003064:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789762199.003070:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1789762199.003071:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762199.003080:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565750016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762199.003128:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762199/real 1789762199] req@ffff93f844d64a80 x1876699565749632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762199.003150:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565749632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762199.003162:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762199.003166:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762199.003171:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565749632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762199.003194:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565749760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762199.003216:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762199.003228:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762199/real 1789762199] req@ffff93f844d65f80 x1876699565749760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762199.003240:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565749760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762199.003249:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762199.003253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762199.003256:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565749760:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762199.003263:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565749888:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762199.003280:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762199.003290:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762199/real 1789762199] req@ffff93f844d64000 x1876699565749888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762199.003302:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565749888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762199.003311:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762199.003314:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762199.003317:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565749888:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762199.003323:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565750016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762199.003339:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762199.003348:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762199/real 1789762199] req@ffff93f844d66d80 x1876699565750016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762199.003360:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565750016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762199.003367:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762199.003371:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762199.003374:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565750016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762199.066769:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762200.090860:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762201.114856:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762202.138824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762203.162798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762204.122762:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762204.122771:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762204.122775:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762204.122784:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2139 00000100:00080000:2.0:1789762204.122794:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762204.122801:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762204.122839:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565750144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762204.122874:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762204.122877:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762204.122878:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762204.122881:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2139 00000100:00080000:2.0:1789762204.122883:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762204.122886:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762204.122896:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762204.122907:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565750272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762204.122915:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762204.122918:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762204.122920:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762204.122927:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2139 00000100:00100000:0.0:1789762204.122929:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565750144:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762204.122930:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762204.122942:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762204.122946:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762204.122953:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565750400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762204.122960:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762204.122963:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762204.122964:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762204.122967:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2139 00000100:00080000:2.0:1789762204.122977:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762204.122980:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762204.122984:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762204.122993:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565750528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762204.123013:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762204.123068:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762204/real 1789762204] req@ffff93f8517c2d80 x1876699565750144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762204.123086:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565750144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762204.123096:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762204.123099:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762204.123102:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565750144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762204.123128:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565750272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762204.123150:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762204.123164:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762204/real 1789762204] req@ffff93f8517c2a00 x1876699565750272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762204.123186:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565750272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762204.123195:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762204.123197:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762204.123200:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565750272:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762204.123205:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565750400:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762204.123223:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762204.123231:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762204/real 1789762204] req@ffff93f8517c3100 x1876699565750400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762204.123244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565750400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762204.123252:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762204.123256:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762204.123259:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565750400:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762204.123266:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565750528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762204.123289:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762204.123299:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762204/real 1789762204] req@ffff93f8517c1180 x1876699565750528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762204.123309:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565750528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762204.123324:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762204.123328:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762204.123331:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565750528:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762204.186743:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762205.210822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762206.234837:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762207.258792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762208.282882:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762209.242768:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762209.242783:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762209.242789:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762209.242802:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2144 00000100:00080000:2.0:1789762209.242827:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762209.242836:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762209.242874:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565750656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762209.242903:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762209.242906:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762209.242908:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762209.242911:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2144 00000100:00080000:2.0:1789762209.242914:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762209.242917:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762209.242926:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762209.242937:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565750784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762209.242951:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565750656:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762209.242951:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762209.242962:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762209.242966:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762209.242972:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2144 00000100:00080000:2.0:1789762209.242976:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762209.242981:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762209.242990:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762209.243006:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565750912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762209.243019:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762209.243023:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762209.243026:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762209.243031:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2144 00000100:00080000:2.0:1789762209.243035:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762209.243039:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762209.243044:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789762209.243052:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762209.243060:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565751040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762209.243107:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762209/real 1789762209] req@ffff93f8517c0380 x1876699565750656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762209.243130:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565750656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762209.243143:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762209.243148:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762209.243152:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565750656:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762209.243181:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565750784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762209.243208:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762209.243225:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762209/real 1789762209] req@ffff93f8517c1500 x1876699565750784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762209.243239:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565750784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762209.243248:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762209.243253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762209.243258:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565750784:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762209.243267:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565750912:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762209.243296:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762209.243314:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762209/real 1789762209] req@ffff93f8517c2680 x1876699565750912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762209.243330:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565750912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762209.243341:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762209.243345:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762209.243349:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565750912:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762209.243356:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565751040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762209.243379:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762209.243391:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762209/real 1789762209] req@ffff93f8517c0a80 x1876699565751040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762209.243400:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565751040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762209.243406:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762209.243409:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762209.243411:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565751040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762209.306793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762210.330827:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762211.354766:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762212.378854:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762213.402905:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762214.362747:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762214.362757:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762214.362761:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762214.362782:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2149 00000100:00080000:3.0:1789762214.362791:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762214.362796:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762214.362837:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565751168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762214.362863:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762214.362866:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762214.362867:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762214.362870:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2149 00000100:00080000:3.0:1789762214.362873:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762214.362875:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762214.362884:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762214.362900:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565751296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762214.362910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762214.362913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762214.362914:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762214.362923:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2149 00000100:00100000:0.0:1789762214.362929:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565751168:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762214.362929:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762214.362943:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762214.362950:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762214.362964:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565751424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762214.362971:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762214.362974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762214.362976:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762214.362978:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2149 00000100:00080000:3.0:1789762214.362981:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762214.362984:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762214.362988:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762214.363002:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565751552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762214.363023:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762214.363076:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762214/real 1789762214] req@ffff93f844d64e00 x1876699565751168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762214.363092:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565751168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762214.363102:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762214.363106:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762214.363110:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565751168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762214.363137:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565751296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762214.363155:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762214.363164:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762214/real 1789762214] req@ffff93f844d65c00 x1876699565751296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762214.363172:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565751296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762214.363183:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762214.363186:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762214.363188:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565751296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762214.363192:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565751424:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762214.363204:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762214.363210:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762214/real 1789762214] req@ffff93f844d66a00 x1876699565751424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762214.363218:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565751424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762214.363224:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762214.363225:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762214.363227:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565751424:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762214.363231:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565751552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762214.363250:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762214.363261:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762214/real 1789762214] req@ffff93f844d67100 x1876699565751552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762214.363272:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565751552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762214.363281:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762214.363285:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762214.363289:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565751552:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762214.426786:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762215.450870:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762216.474811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762217.498851:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762218.522834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762219.482761:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762219.482774:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762219.482780:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762219.482793:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2154 00000100:00080000:3.0:1789762219.482806:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762219.482816:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762219.482863:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565751680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762219.482899:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762219.482907:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762219.482909:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762219.482913:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2154 00000100:00080000:3.0:1789762219.482917:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762219.482922:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762219.482933:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762219.482948:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565751808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762219.482953:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565751680:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762219.482968:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762219.482971:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762219.482973:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762219.482979:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2154 00000100:00080000:3.0:1789762219.482983:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762219.482987:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762219.482993:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762219.483007:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565751936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762219.483018:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762219.483022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762219.483024:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762219.483028:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2154 00000100:00080000:3.0:1789762219.483032:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789762219.483039:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762219.483039:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762219.483048:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762219.483060:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565752064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762219.483093:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762219/real 1789762219] req@ffff93f844d67800 x1876699565751680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762219.483112:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565751680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762219.483121:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762219.483125:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762219.483128:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565751680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762219.483152:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565751808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762219.483168:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762219.483178:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762219/real 1789762219] req@ffff93f844d67480 x1876699565751808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762219.483186:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565751808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762219.483192:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762219.483194:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762219.483196:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565751808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762219.483200:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565751936:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762219.483213:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762219.483220:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762219/real 1789762219] req@ffff93f844d65880 x1876699565751936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762219.483227:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565751936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762219.483232:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762219.483234:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762219.483236:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565751936:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762219.483239:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565752064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762219.483250:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762219.483256:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762219/real 1789762219] req@ffff93f844d66680 x1876699565752064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762219.483263:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565752064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762219.483268:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762219.483270:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762219.483272:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565752064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762219.546888:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762220.570792:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762221.594787:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762222.618805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762223.642879:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762224.602749:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762224.602757:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762224.602760:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762224.602770:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2159 00000100:00080000:2.0:1789762224.602778:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762224.602786:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762224.602821:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565752192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762224.602851:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762224.602855:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762224.602857:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762224.602862:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2159 00000100:00080000:2.0:1789762224.602866:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762224.602872:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762224.602883:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762224.602892:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565752192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762224.602897:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565752320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762224.602907:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762224.602910:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762224.602911:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762224.602915:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2159 00000100:00080000:2.0:1789762224.602918:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762224.602921:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762224.602924:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762224.602933:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565752448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762224.602940:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762224.602943:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762224.602944:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762224.602946:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2159 00000100:00080000:2.0:1789762224.602950:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789762224.602954:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762224.602955:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762224.602960:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762224.602968:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565752576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762224.602999:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762224/real 1789762224] req@ffff93f8517c2300 x1876699565752192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762224.603020:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565752192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762224.603031:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762224.603038:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762224.603042:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565752192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762224.603066:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565752320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762224.603084:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762224.603094:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762224/real 1789762224] req@ffff93f8517c1f80 x1876699565752320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762224.603102:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565752320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762224.603108:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762224.603110:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762224.603112:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565752320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762224.603116:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565752448:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762224.603127:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762224.603133:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762224/real 1789762224] req@ffff93f8517c3b80 x1876699565752448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762224.603140:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565752448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762224.603149:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762224.603151:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762224.603154:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565752448:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762224.603157:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565752576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762224.603167:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762224.603183:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762224/real 1789762224] req@ffff93f8517c0000 x1876699565752576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762224.603190:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565752576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762224.603196:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762224.603198:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762224.603199:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565752576:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762224.666804:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762225.690834:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762226.714836:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762227.738846:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762228.762824:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762229.722801:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762229.722813:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762229.722820:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762229.722832:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2164 00000100:00080000:3.0:1789762229.722842:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762229.722850:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762229.722897:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565752704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762229.722933:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762229.722937:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762229.722939:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762229.722943:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2164 00000100:00080000:3.0:1789762229.722948:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762229.722952:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762229.722961:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762229.722975:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565752832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762229.722985:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762229.722988:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762229.722990:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762229.722993:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2164 00000100:00080000:3.0:1789762229.722997:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762229.723000:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762229.723005:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762229.723018:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565752960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762229.723027:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762229.723030:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762229.723032:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762229.723036:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2164 00000100:00080000:3.0:1789762229.723039:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762229.723042:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762229.723047:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762229.723057:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565753088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762229.723242:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565752704:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1789762229.723297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762229.723347:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762229/real 1789762229] req@ffff93f844d64700 x1876699565752704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762229.723366:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565752704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762229.723374:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762229.723378:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762229.723382:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565752704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762229.723387:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565752832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762229.723405:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762229.723415:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762229/real 1789762229] req@ffff93f844d67b80 x1876699565752832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762229.723422:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565752832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762229.723428:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762229.723430:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762229.723432:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565752832:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762229.723436:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565752960:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762229.723447:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762229.723453:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762229/real 1789762229] req@ffff93f844d66680 x1876699565752960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762229.723460:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565752960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762229.723465:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762229.723467:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762229.723469:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565752960:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762229.723472:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565753088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762229.723482:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762229.723488:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762229/real 1789762229] req@ffff93f844d65880 x1876699565753088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762229.723495:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565753088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762229.723500:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762229.723502:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762229.723504:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565753088:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762229.786831:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762230.810780:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762231.834865:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762232.858843:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762233.882842:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762234.842790:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762234.842800:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762234.842805:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762234.842815:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2169 00000100:00080000:0.0:1789762234.842825:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762234.842832:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762234.842871:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565753216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762234.842909:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762234.842924:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762234.842926:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762234.842931:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2169 00000100:00080000:0.0:1789762234.842935:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762234.842940:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762234.842950:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762234.842966:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565753344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762234.842990:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762234.842993:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762234.842995:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762234.843000:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2169 00000100:00080000:0.0:1789762234.843005:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762234.843009:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762234.843015:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762234.843027:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565753472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762234.843039:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762234.843043:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762234.843046:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789762234.843055:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565753216:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762234.843057:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2169 00000100:00080000:0.0:1789762234.843068:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762234.843076:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762234.843083:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762234.843099:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565753600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762234.843189:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762234.843277:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762234/real 1789762234] req@ffff93f850d40e00 x1876699565753216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762234.843310:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565753216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762234.843330:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762234.843337:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762234.843344:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565753216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762234.843376:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565753344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762234.843406:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762234.843440:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762234/real 1789762234] req@ffff93f850d41c00 x1876699565753344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762234.843459:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565753344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762234.843471:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762234.843477:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762234.843500:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565753344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762234.843542:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565753472:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762234.843587:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762234.843599:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762234/real 1789762234] req@ffff93f850d42680 x1876699565753472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762234.843611:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565753472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762234.843641:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762234.843720:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762234.843725:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565753472:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762234.843733:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565753600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762234.843784:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762234.843802:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762234/real 1789762234] req@ffff93f850d40000 x1876699565753600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762234.843833:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565753600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762234.843841:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762234.843844:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762234.843848:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565753600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762234.906881:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762235.930834:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762236.954844:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762237.978869:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762239.002803:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762239.962769:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762239.962781:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762239.962786:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762239.962798:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2174 00000100:00080000:1.0:1789762239.962812:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762239.962821:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762239.962868:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565753728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762239.962905:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762239.962909:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762239.962911:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762239.962916:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2174 00000100:00080000:1.0:1789762239.962920:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762239.962924:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762239.962935:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762239.962949:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565753856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762239.962962:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762239.962966:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762239.962968:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789762239.962972:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565753728:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762239.962974:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2174 00000100:00080000:1.0:1789762239.962982:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762239.962987:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762239.962992:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762239.963006:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565753984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762239.963019:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762239.963023:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762239.963025:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762239.963029:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2174 00000100:00080000:1.0:1789762239.963034:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762239.963038:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762239.963043:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762239.963054:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565754112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762239.963064:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762239.963129:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762239/real 1789762239] req@ffff93f844cc9180 x1876699565753728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762239.963152:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565753728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762239.963165:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762239.963170:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762239.963175:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565753728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762239.963206:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565753856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762239.963227:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762239.963238:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762239/real 1789762239] req@ffff93f844cc8a80 x1876699565753856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762239.963247:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565753856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762239.963253:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762239.963255:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762239.963258:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565753856:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762239.963261:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565753984:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762239.963273:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762239.963280:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762239/real 1789762239] req@ffff93f844cc9880 x1876699565753984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762239.963288:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565753984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762239.963298:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762239.963300:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762239.963302:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565753984:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762239.963305:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565754112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762239.963316:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762239.963321:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762239/real 1789762239] req@ffff93f844cc9f80 x1876699565754112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762239.963329:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565754112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762239.963335:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762239.963337:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762239.963338:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565754112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762240.026869:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762241.050912:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762242.074862:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762243.098828:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762244.122840:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762245.082783:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762245.082800:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762245.082807:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762245.082841:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2180 00000100:00080000:1.0:1789762245.082859:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762245.082875:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762245.082936:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565754240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762245.082982:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762245.082986:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762245.082989:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762245.083006:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2180 00000100:00080000:1.0:1789762245.083011:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762245.083016:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762245.083028:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565754240:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762245.083029:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762245.083055:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565754368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762245.083066:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762245.083070:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762245.083072:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762245.083077:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2180 00000100:00080000:1.0:1789762245.083096:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762245.083099:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762245.083103:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762245.083113:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565754496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762245.083123:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762245.083127:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762245.083129:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789762245.083136:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762245.083137:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2180 00000100:00080000:1.0:1789762245.083146:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762245.083152:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762245.083159:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762245.083177:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565754624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762245.083190:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762245/real 1789762245] req@ffff93f844cca300 x1876699565754240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762245.083208:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565754240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762245.083219:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762245.083223:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762245.083227:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565754240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762245.083251:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565754368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762245.083270:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762245.083281:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762245/real 1789762245] req@ffff93f844cc8380 x1876699565754368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762245.083290:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565754368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762245.083299:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762245.083304:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762245.083308:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565754368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762245.083316:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565754496:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762245.083347:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762245.083366:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762245/real 1789762245] req@ffff93f844cc8000 x1876699565754496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762245.083377:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565754496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762245.083384:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762245.083387:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762245.083390:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565754496:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762245.083395:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565754624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762245.083415:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762245.083427:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762245/real 1789762245] req@ffff93f844ccbb80 x1876699565754624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762245.083439:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565754624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762245.083446:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762245.083449:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762245.083454:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565754624:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762245.146823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762246.170770:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762247.194831:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762248.218755:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762249.242833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762250.202764:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762250.202777:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762250.202783:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762250.202797:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2185 00000100:00080000:1.0:1789762250.202809:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762250.202817:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762250.202867:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565754752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762250.202901:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762250.202906:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762250.202908:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762250.202914:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2185 00000100:00080000:1.0:1789762250.202918:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762250.202924:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762250.202936:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762250.202948:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565754752:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762250.202959:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565754880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762250.202974:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762250.202978:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762250.202980:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762250.202985:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2185 00000100:00080000:1.0:1789762250.202991:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762250.202996:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762250.203013:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762250.203029:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565755008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762250.203043:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762250.203047:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762250.203050:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762250.203055:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2185 00000100:00080000:1.0:1789762250.203061:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789762250.203065:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565754880:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1789762250.203071:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762250.203080:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762250.203097:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565755136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762250.203098:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565755008:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762250.203128:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565755136:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762250.266763:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762251.290799:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762252.314778:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789762253.274990:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789762253.275065:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789762253.275093:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762253.275174:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762253.275188:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762253.275207:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789762253.275341:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762250/real 1789762253] req@ffff93f844cc8700 x1876699565755136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762253.275380:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565755136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762253.275398:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762253.275406:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762253.275413:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565755136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762253.275426:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762250/real 1789762253] req@ffff93f844ccb800 x1876699565754880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762253.275445:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565754880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762253.275458:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762253.275464:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762253.275469:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565754880:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762253.275506:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762250/real 1789762253] req@ffff93f844ccb480 x1876699565755008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762253.275529:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565755008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762253.275541:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762253.275547:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762253.275555:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565755008:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762253.275565:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762250/real 1789762253] req@ffff93f844ccaa00 x1876699565754752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762253.275584:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565754752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762253.275596:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762253.275616:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762253.275621:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565754752:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1789762253.338812:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762254.362856:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762255.322727:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762255.322740:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762255.322746:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762255.322760:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2190 00000100:00080000:1.0:1789762255.322775:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762255.322786:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762255.322839:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565755264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762255.322877:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762255.322882:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762255.322884:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762255.322889:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2190 00000100:00080000:1.0:1789762255.322893:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762255.322898:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762255.322909:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762255.322925:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565755392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762255.322932:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565755264:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762255.322936:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762255.322939:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762255.322941:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762255.322951:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2190 00000100:00080000:1.0:1789762255.322955:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762255.322959:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762255.322963:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762255.322973:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565755520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762255.322983:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762255.322987:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762255.322988:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762255.322992:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2190 00000100:00080000:1.0:1789762255.323003:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762255.323008:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789762255.323012:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789762255.323013:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762255.323023:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565755648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762255.323066:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762255/real 1789762255] req@ffff93f844ccb100 x1876699565755264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762255.323087:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565755264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762255.323128:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762255.323134:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762255.323139:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565755264:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762255.323189:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565755392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762255.323216:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762255.323231:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762255/real 1789762255] req@ffff93f844cc9c00 x1876699565755392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762255.323251:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565755392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762255.323260:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762255.323263:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762255.323266:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565755392:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762255.323272:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565755520:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762255.323295:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762255.323308:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762255/real 1789762255] req@ffff93f844cca680 x1876699565755520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762255.323321:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565755520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762255.323329:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762255.323332:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762255.323336:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565755520:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762255.323342:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565755648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762255.323362:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762255.323374:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762255/real 1789762255] req@ffff93f844cc8e00 x1876699565755648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762255.323386:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565755648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762255.323401:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762255.323405:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762255.323409:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565755648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762255.386831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762256.410826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762257.434827:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762258.458895:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762259.482895:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762260.442738:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762260.442748:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762260.442752:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762260.442762:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2195 00000100:00080000:1.0:1789762260.442772:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762260.442779:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762260.442819:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565755776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762260.442848:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762260.442852:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762260.442854:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762260.442859:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2195 00000100:00080000:1.0:1789762260.442863:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762260.442867:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762260.442878:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762260.442898:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565755776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762260.442905:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565755904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762260.442925:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762260.442928:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762260.442931:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762260.442935:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2195 00000100:00080000:1.0:1789762260.442939:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762260.442944:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762260.442950:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762260.442964:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565756032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762260.442972:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762260.442977:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:0.0:1789762260.442978:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762260.442979:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762260.442985:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2195 00000100:00080000:1.0:1789762260.442989:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762260.442992:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762260.442997:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762260.443005:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565756160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762260.443030:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762260/real 1789762260] req@ffff93f844ccad80 x1876699565755776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762260.443048:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565755776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762260.443058:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762260.443062:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762260.443066:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565755776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762260.443092:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565755904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762260.443116:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762260.443130:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762260/real 1789762260] req@ffff93f844cc9500 x1876699565755904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762260.443142:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565755904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762260.443151:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762260.443155:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762260.443159:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565755904:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762260.443165:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565756032:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762260.443186:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762260.443199:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762260/real 1789762260] req@ffff93f844cc8e00 x1876699565756032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762260.443212:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565756032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762260.443226:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762260.443230:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762260.443233:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565756032:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762260.443240:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565756160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762260.443262:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762260.443273:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762260/real 1789762260] req@ffff93f844cca680 x1876699565756160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762260.443285:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565756160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762260.443293:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762260.443296:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762260.443299:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565756160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762260.506841:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762261.530808:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762262.554886:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762263.578758:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762264.602776:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762265.562798:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762265.562814:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762265.562821:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762265.562835:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2200 00000100:00080000:1.0:1789762265.562851:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762265.562861:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762265.562913:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565756288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762265.562947:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762265.562953:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762265.562957:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762265.562964:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2200 00000100:00080000:1.0:1789762265.562971:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762265.562978:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762265.562991:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762265.562993:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565756288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762265.563013:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565756416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762265.563029:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762265.563033:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762265.563036:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762265.563041:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2200 00000100:00080000:1.0:1789762265.563059:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762265.563082:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789762265.563086:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789762265.563088:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762265.563097:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565756544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762265.563107:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762265.563111:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762265.563113:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762265.563116:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2200 00000100:00080000:1.0:1789762265.563120:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762265.563124:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762265.563129:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762265.563139:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565756672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762265.563140:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762265/real 1789762265] req@ffff93f844cc9c00 x1876699565756288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762265.563159:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565756288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762265.563171:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762265.563175:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762265.563180:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565756288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762265.563204:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565756416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762265.563225:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762265.563235:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762265/real 1789762265] req@ffff93f844ccb100 x1876699565756416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762265.563244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565756416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762265.563267:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762265.563270:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762265.563274:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565756416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762265.563279:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565756544:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762265.563298:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762265.563308:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762265/real 1789762265] req@ffff93f844ccaa00 x1876699565756544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762265.563327:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565756544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762265.563335:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762265.563338:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762265.563341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565756544:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762265.563345:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565756672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762265.563376:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762265.563404:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762265/real 1789762265] req@ffff93f844ccb480 x1876699565756672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762265.563421:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565756672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762265.563455:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762265.563459:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762265.563463:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565756672:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762265.626847:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762266.650806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762267.674917:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762268.698854:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762269.722862:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762270.682800:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762270.682816:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762270.682823:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762270.682837:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2205 00000100:00080000:1.0:1789762270.682852:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762270.682862:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762270.682913:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565756800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762270.682953:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762270.682958:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762270.682960:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762270.682966:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2205 00000100:00080000:1.0:1789762270.682971:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762270.682976:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762270.682996:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565756800:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762270.683000:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762270.683018:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565756928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762270.683031:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762270.683035:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762270.683037:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762270.683042:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2205 00000100:00080000:1.0:1789762270.683061:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789762270.683070:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762270.683075:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762270.683085:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762270.683100:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565757056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762270.683113:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762270.683118:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:0.0:1789762270.683119:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762270/real 1789762270] req@ffff93f844ccb800 x1876699565756800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762270.683123:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762270.683134:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2205 00000100:00080000:1.0:1789762270.683139:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789762270.683145:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565756800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762270.683146:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762270.683157:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00080000:0.0:1789762270.683161:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762270.683166:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762270.683173:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565756800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762270.683175:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565757184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762270.683197:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565756928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762270.683215:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762270.683225:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762270/real 1789762270] req@ffff93f844cc8700 x1876699565756928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762270.683233:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565756928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762270.683239:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762270.683242:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762270.683244:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565756928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762270.683248:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565757056:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762270.683259:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762270.683265:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762270/real 1789762270] req@ffff93f844ccbb80 x1876699565757056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762270.683273:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565757056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762270.683279:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762270.683281:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762270.683283:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565757056:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762270.683286:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565757184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762270.683297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762270.683303:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762270/real 1789762270] req@ffff93f844cc8000 x1876699565757184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762270.683311:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565757184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762270.683316:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762270.683318:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762270.683320:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565757184:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762270.746852:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762271.770869:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762272.794783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762273.818823:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762274.842780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762275.802794:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762275.802807:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762275.802813:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762275.802825:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2210 00000100:00080000:3.0:1789762275.802839:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762275.802848:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762275.802903:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565757312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762275.802942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762275.802947:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762275.802949:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762275.802954:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2210 00000100:00080000:3.0:1789762275.802958:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762275.802962:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762275.802971:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762275.802985:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565757440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762275.802995:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762275.802998:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762275.803000:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762275.803004:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2210 00000100:00080000:3.0:1789762275.803007:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762275.803010:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762275.803015:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762275.803024:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565757312:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762275.803027:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565757568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762275.803065:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762275.803068:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762275.803069:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762275.803073:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2210 00000100:00080000:3.0:1789762275.803077:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762275.803080:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762275.803084:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762275.803093:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565757696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762275.803141:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762275.803229:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762275/real 1789762275] req@ffff93f844d67480 x1876699565757312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762275.803246:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565757312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762275.803256:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762275.803260:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762275.803263:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565757312:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762275.803270:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565757440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762275.803297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762275.803344:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762275/real 1789762275] req@ffff93f844d67800 x1876699565757440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762275.803369:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565757440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762275.803378:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762275.803382:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762275.803386:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565757440:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762275.803437:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565757568:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762275.803457:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762275.803466:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762275/real 1789762275] req@ffff93f844d67100 x1876699565757568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762275.803475:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565757568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762275.803482:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762275.803484:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762275.803486:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565757568:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762275.803491:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565757696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762275.803503:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762275.803510:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762275/real 1789762275] req@ffff93f844d66a00 x1876699565757696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762275.803524:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565757696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762275.803530:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762275.803532:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762275.803534:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565757696:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762275.866846:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762276.890740:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762277.914865:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762278.938815:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762279.962844:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762280.922736:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762280.922751:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762280.922757:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762280.922771:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2215 00000100:00080000:3.0:1789762280.922784:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762280.922794:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762280.922839:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565757824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762280.922870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762280.922875:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762280.922877:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762280.922883:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2215 00000100:00080000:3.0:1789762280.922889:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762280.922895:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762280.922906:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762280.922921:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565757952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762280.922931:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565757824:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762280.922950:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762280.922954:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762280.922957:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762280.922964:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2215 00000100:00080000:3.0:1789762280.922969:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762280.922974:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762280.922981:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762280.922998:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565758080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762280.923011:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762280.923058:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762280/real 1789762280] req@ffff93f844d65c00 x1876699565757824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762280.923074:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565757824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762280.923084:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762280.923088:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762280.923093:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565757824:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762280.923094:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762280.923102:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762280.923105:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762280.923112:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2215 00000100:00080000:3.0:1789762280.923118:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789762280.923122:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565757952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762280.923140:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762280.923149:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762280/real 1789762280] req@ffff93f844d64e00 x1876699565757952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762280.923158:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565757952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762280.923164:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762280.923167:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762280.923169:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565757952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762280.923173:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565758080:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762280.923185:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762280.923191:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762280/real 1789762280] req@ffff93f844d66d80 x1876699565758080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762280.923199:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565758080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762280.923206:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762280.923208:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762280.923210:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565758080:192.168.203.147@tcp:8:kworker.0 00000100:00080000:3.0:1789762280.923264:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762280.923326:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762280.923344:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565758208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762280.923393:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565758208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762280.923412:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762280.923421:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762280/real 1789762280] req@ffff93f844d64000 x1876699565758208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762280.923430:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565758208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762280.923436:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762280.923438:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762280.923440:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565758208:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762280.986767:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762282.010918:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762283.034793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762284.058802:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762285.082806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762286.042745:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762286.042756:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762286.042760:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762286.042770:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2221 00000100:00080000:1.0:1789762286.042779:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762286.042785:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762286.042827:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565758336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762286.042855:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762286.042858:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762286.042859:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762286.042863:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2221 00000100:00080000:1.0:1789762286.042866:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762286.042871:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762286.042880:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762286.042884:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565758336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762286.042894:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565758464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762286.042904:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762286.042906:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762286.042908:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762286.042910:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2221 00000100:00080000:1.0:1789762286.042913:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762286.042915:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762286.042918:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762286.042926:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565758592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762286.042934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762286.042936:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762286.042937:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762286.042940:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2221 00000100:00080000:1.0:1789762286.042943:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789762286.042947:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762286.042950:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762286.042955:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762286.042972:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565758720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762286.042985:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762286/real 1789762286] req@ffff93f844cc8380 x1876699565758336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762286.043000:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565758336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762286.043008:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762286.043011:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762286.043014:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565758336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762286.043033:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565758464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762286.043047:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762286.043055:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762286/real 1789762286] req@ffff93f844cca300 x1876699565758464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762286.043063:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565758464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762286.043077:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762286.043081:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762286.043084:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565758464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762286.043090:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565758592:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762286.043111:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762286.043123:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762286/real 1789762286] req@ffff93f844cc9f80 x1876699565758592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762286.043135:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565758592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762286.043144:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762286.043147:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762286.043151:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565758592:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762286.043156:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565758720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762286.043172:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762286.043179:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762286/real 1789762286] req@ffff93f844cc9880 x1876699565758720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762286.043206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565758720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762286.043211:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762286.043213:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762286.043216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565758720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762286.106825:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762287.130950:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762288.154792:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762289.178829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762290.202828:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762291.162766:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762291.162778:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762291.162783:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762291.162797:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2226 00000100:00080000:1.0:1789762291.162810:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762291.162818:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762291.162866:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565758848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762291.162901:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762291.162905:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762291.162907:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762291.162918:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2226 00000100:00080000:1.0:1789762291.162922:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762291.162925:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762291.162936:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762291.162950:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565758976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762291.162966:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762291.162969:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762291.162971:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789762291.162977:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565758848:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762291.162978:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2226 00000100:00080000:1.0:1789762291.162985:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762291.162990:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762291.162994:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762291.163005:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565759104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762291.163013:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762291.163017:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762291.163019:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762291.163022:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2226 00000100:00080000:1.0:1789762291.163025:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762291.163028:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762291.163031:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762291.163039:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565759232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762291.163067:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762291.163185:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762291/real 1789762291] req@ffff93f844cc8a80 x1876699565758848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762291.163238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565758848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762291.163252:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762291.163258:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762291.163264:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565758848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762291.163300:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565758976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762291.163333:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762291.163350:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762291/real 1789762291] req@ffff93f844cc9180 x1876699565758976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762291.163362:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565758976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762291.163370:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762291.163373:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762291.163376:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565758976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762291.163381:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565759104:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762291.163400:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762291.163409:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762291/real 1789762291] req@ffff93f844cc9880 x1876699565759104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762291.163419:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565759104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762291.163425:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762291.163428:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762291.163432:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565759104:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762291.163440:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565759232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762291.163465:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762291.163480:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762291/real 1789762291] req@ffff93f844cc9f80 x1876699565759232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762291.163508:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565759232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762291.163518:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762291.163522:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762291.163526:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565759232:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762291.226930:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762292.250767:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762293.274852:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762294.298768:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762295.322932:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762296.283146:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762296.283184:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762296.283191:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762296.283207:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2231 00000100:00080000:3.0:1789762296.283231:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762296.283244:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762296.283302:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565759360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762296.283345:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762296.283351:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762296.283354:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762296.283361:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2231 00000100:00080000:3.0:1789762296.283368:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762296.283374:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762296.283389:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762296.283413:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565759488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762296.283432:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762296.283438:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762296.283441:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762296.283448:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2231 00000100:00080000:3.0:1789762296.283455:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762296.283460:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762296.283469:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762296.283492:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565759616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762296.283513:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762296.283519:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762296.283523:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762296.283530:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2231 00000100:00080000:3.0:1789762296.283537:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762296.283545:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762296.283554:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762296.283575:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565759744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762296.283638:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565759360:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1789762296.283804:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762296.283874:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762296/real 1789762296] req@ffff93f844d65f80 x1876699565759360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762296.283903:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565759360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762296.283919:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762296.283926:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762296.283933:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565759360:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762296.283944:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565759488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762296.283984:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762296.284006:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762296/real 1789762296] req@ffff93f844d64a80 x1876699565759488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762296.284028:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565759488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762296.284041:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762296.284044:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762296.284048:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565759488:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762296.284055:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565759616:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762296.284081:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762296.284093:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762296/real 1789762296] req@ffff93f844d64380 x1876699565759616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762296.284105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565759616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762296.284113:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762296.284116:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762296.284119:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565759616:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762296.284134:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565759744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762296.284182:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762296.284196:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762296/real 1789762296] req@ffff93f844d65500 x1876699565759744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762296.284218:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565759744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762296.284232:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762296.284237:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762296.284243:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565759744:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762296.346889:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762297.370970:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762298.394895:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762299.418802:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762300.442916:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762301.402765:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762301.402776:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762301.402780:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762301.402790:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2236 00000100:00080000:3.0:1789762301.402800:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762301.402807:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762301.402851:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565759872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762301.402878:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762301.402881:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762301.402882:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762301.402886:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2236 00000100:00080000:3.0:1789762301.402888:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762301.402890:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762301.402899:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762301.402910:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565760000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762301.402919:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762301.402922:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762301.402924:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762301.402927:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2236 00000100:00080000:3.0:1789762301.402930:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762301.402944:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762301.402948:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762301.402952:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565759872:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762301.402960:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565760128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762301.402973:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762301.402975:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762301.402976:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762301.402980:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2236 00000100:00080000:3.0:1789762301.402983:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762301.402987:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762301.402991:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762301.403000:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565760256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762301.403037:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762301.403102:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762301/real 1789762301] req@ffff93f844d65500 x1876699565759872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762301.403152:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565759872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762301.403165:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762301.403170:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762301.403175:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565759872:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762301.403203:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565760000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762301.403231:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762301.403256:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762301/real 1789762301] req@ffff93f844d64380 x1876699565760000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762301.403270:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565760000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762301.403280:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762301.403284:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762301.403287:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565760000:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762301.403294:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565760128:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762301.403314:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762301.403323:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762301/real 1789762301] req@ffff93f844d64a80 x1876699565760128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762301.403331:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565760128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762301.403336:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762301.403339:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762301.403341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565760128:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762301.403345:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565760256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762301.403357:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762301.403363:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762301/real 1789762301] req@ffff93f844d65f80 x1876699565760256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762301.403370:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565760256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762301.403375:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762301.403377:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762301.403379:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565760256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762301.466830:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762302.490801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762303.514834:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762304.538805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762305.562822:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762306.522763:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762306.522771:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762306.522775:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762306.522787:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2241 00000100:00080000:0.0:1789762306.522798:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762306.522812:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762306.522846:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565760384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762306.522892:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762306.522896:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762306.522898:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762306.522902:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2241 00000100:00080000:0.0:1789762306.522905:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762306.522909:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762306.522919:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762306.522930:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565760512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762306.522939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762306.522942:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762306.522944:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762306.522947:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2241 00000100:00080000:0.0:1789762306.522949:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762306.522952:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762306.522957:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762306.522962:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565760384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762306.522969:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565760640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762306.522980:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762306.522983:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762306.522986:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762306.522989:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2241 00000100:00080000:0.0:1789762306.522993:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762306.522998:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762306.523003:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762306.523018:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565760768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789762306.523043:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762306.523128:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762306/real 1789762306] req@ffff93f850d41180 x1876699565760384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762306.523148:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565760384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762306.523156:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762306.523158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762306.523161:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565760384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762306.523167:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565760512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762306.523184:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762306.523193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762306/real 1789762306] req@ffff93f850d43100 x1876699565760512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762306.523200:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565760512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762306.523206:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762306.523208:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762306.523210:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565760512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762306.523228:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565760640:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762306.523239:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762306.523245:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762306/real 1789762306] req@ffff93f850d41f80 x1876699565760640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762306.523252:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565760640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762306.523262:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762306.523264:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762306.523266:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565760640:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762306.523269:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565760768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762306.523278:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762306.523283:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762306/real 1789762306] req@ffff93f850d41500 x1876699565760768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762306.523290:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565760768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762306.523296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762306.523297:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762306.523299:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565760768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762306.586787:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762307.610809:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762308.634863:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762309.658904:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762310.682831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762311.642763:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762311.642771:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762311.642775:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762311.642784:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2246 00000100:00080000:3.0:1789762311.642792:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762311.642798:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762311.642828:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565760896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762311.642858:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762311.642860:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762311.642862:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762311.642864:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2246 00000100:00080000:3.0:1789762311.642871:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762311.642874:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762311.642881:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762311.642890:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565761024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762311.642897:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762311.642899:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762311.642901:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762311.642903:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2246 00000100:00080000:3.0:1789762311.642905:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762311.642907:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762311.642910:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762311.642916:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565760896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762311.642922:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565761152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762311.642931:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762311.642933:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762311.642934:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762311.642937:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2246 00000100:00080000:3.0:1789762311.642940:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762311.642942:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762311.642945:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762311.642952:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565761280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762311.642980:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762311.643021:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762311/real 1789762311] req@ffff93f844d66300 x1876699565760896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762311.643036:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565760896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762311.643056:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762311.643060:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762311.643063:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565760896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762311.643070:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565761024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762311.643110:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762311.643124:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762311/real 1789762311] req@ffff93f844d65180 x1876699565761024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762311.643139:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565761024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762311.643146:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762311.643149:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762311.643151:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565761024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762311.643173:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565761152:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762311.643190:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762311.643197:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762311/real 1789762311] req@ffff93f844d65f80 x1876699565761152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762311.643205:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565761152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762311.643211:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762311.643213:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762311.643215:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565761152:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762311.643218:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565761280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762311.643228:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762311.643234:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762311/real 1789762311] req@ffff93f844d64a80 x1876699565761280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762311.643241:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565761280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762311.643250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762311.643252:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762311.643254:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565761280:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762311.706795:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762312.730797:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762313.754765:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762314.778819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762315.802819:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762316.762720:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762316.762732:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762316.762737:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762316.762748:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2251 00000100:00080000:3.0:1789762316.762760:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762316.762768:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762316.762806:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565761408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762316.762830:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762316.762834:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762316.762835:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762316.762839:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2251 00000100:00080000:3.0:1789762316.762841:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762316.762844:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762316.762853:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762316.762864:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565761536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762316.762871:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762316.762875:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762316.762876:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762316.762879:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2251 00000100:00080000:3.0:1789762316.762881:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762316.762884:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762316.762887:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762316.762897:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565761664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762316.762905:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762316.762908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762316.762910:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762316.762913:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2251 00000100:00080000:3.0:1789762316.762917:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762316.762921:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762316.762924:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762316.762933:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565761792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762316.762949:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565761408:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762316.763061:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565761536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762316.763085:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565761664:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762316.763107:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565761792:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762316.826815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762317.850789:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762318.874813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789762319.835127:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789762319.835223:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789762319.835248:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762319.835326:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762319.835340:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762319.835352:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789762319.835467:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762316/real 1789762319] req@ffff93f844d66d80 x1876699565761792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762319.835504:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565761792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762319.835521:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762319.835528:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762319.835535:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565761792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762319.835547:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762316/real 1789762319] req@ffff93f844d64380 x1876699565761408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762319.835564:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565761408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762319.835575:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762319.835580:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762319.835584:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565761408:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762319.835593:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762316/real 1789762319] req@ffff93f844d65500 x1876699565761536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762319.835608:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565761536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762319.835620:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762319.835624:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762319.835629:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565761536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762319.835638:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762316/real 1789762319] req@ffff93f844d64000 x1876699565761664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762319.835796:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565761664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762319.835809:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762319.835815:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762319.835819:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565761664:192.168.203.147@tcp:8:kworker.0 00010000:00010000:3.0:1789762319.898873:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762320.922867:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762321.882761:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762321.882772:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762321.882776:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762321.882788:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2256 00000100:00080000:0.0:1789762321.882802:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762321.882809:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762321.882854:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565761920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762321.882872:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762321.882875:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762321.882876:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762321.882879:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2256 00000100:00080000:0.0:1789762321.882881:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762321.882884:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762321.882891:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762321.882900:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565762048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762321.882926:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789762321.882927:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565761920:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762321.882934:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762321.882938:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762321.882943:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2256 00000100:00080000:0.0:1789762321.882947:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762321.882951:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762321.882957:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762321.882985:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565762176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762321.882996:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762321.883000:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762321.883002:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762321.883006:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2256 00000100:00080000:0.0:1789762321.883011:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789762321.883014:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762321.883019:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762321.883025:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762321.883040:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565762304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762321.883076:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762321/real 1789762321] req@ffff93f850d43800 x1876699565761920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762321.883098:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565761920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762321.883109:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762321.883114:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762321.883118:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565761920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762321.883143:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565762048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762321.883185:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762321.883199:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762321/real 1789762321] req@ffff93f850d43480 x1876699565762048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762321.883213:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565762048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762321.883223:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762321.883226:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762321.883229:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565762048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762321.883236:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565762176:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762321.883257:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762321.883269:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762321/real 1789762321] req@ffff93f850d41500 x1876699565762176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762321.883282:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565762176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762321.883290:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762321.883293:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762321.883296:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565762176:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762321.883303:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565762304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762321.883324:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762321.883336:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762321/real 1789762321] req@ffff93f850d41f80 x1876699565762304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762321.883353:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565762304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762321.883361:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762321.883364:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762321.883367:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565762304:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762321.946801:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762322.970837:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762323.994806:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762325.018872:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762326.042794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762327.002820:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762327.002829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762327.002832:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762327.002842:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2261 00000100:00080000:2.0:1789762327.002850:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762327.002857:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762327.002892:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565762432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762327.002912:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762327.002914:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762327.002916:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762327.002919:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2261 00000100:00080000:2.0:1789762327.002922:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762327.002943:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762327.002952:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762327.002953:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565762432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762327.002965:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565762560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762327.002979:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762327.002981:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762327.002983:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762327.002986:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2261 00000100:00080000:2.0:1789762327.002990:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762327.002993:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762327.002996:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762327.003004:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565762688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762327.003011:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:1.0:1789762327.003016:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762327.003017:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762327.003020:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762327.003024:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2261 00000100:00080000:2.0:1789762327.003026:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762327.003029:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762327.003033:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762327.003040:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565762816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762327.003064:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762327/real 1789762327] req@ffff93f8517c0000 x1876699565762432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762327.003081:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565762432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762327.003089:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762327.003092:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762327.003095:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565762432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762327.003114:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565762560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762327.003129:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762327.003137:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762327/real 1789762327] req@ffff93f8517c3b80 x1876699565762560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762327.003145:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565762560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762327.003153:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762327.003156:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762327.003160:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565762560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762327.003167:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565762688:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762327.003188:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762327.003201:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762327/real 1789762327] req@ffff93f8517c1f80 x1876699565762688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762327.003212:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565762688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762327.003218:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762327.003220:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762327.003222:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565762688:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762327.003226:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565762816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762327.003241:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762327.003248:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762327/real 1789762327] req@ffff93f8517c2300 x1876699565762816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762327.003255:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565762816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762327.003261:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762327.003262:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762327.003264:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565762816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762327.066804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762328.090865:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762329.114794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762330.138841:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762331.162815:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762332.122772:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762332.122783:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762332.122789:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762332.122801:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2267 00000100:00080000:1.0:1789762332.122813:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762332.122820:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762332.122863:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565762944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762332.122909:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762332.122914:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762332.122916:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762332.122921:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2267 00000100:00080000:1.0:1789762332.122925:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762332.122931:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762332.122943:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762332.122956:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565763072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762332.122962:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565762944:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762332.122968:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762332.122972:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762332.122973:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762332.122977:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2267 00000100:00080000:1.0:1789762332.122981:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762332.122983:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762332.122987:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762332.122995:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565763200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762332.123003:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762332.123006:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762332.123007:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762332.123009:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2267 00000100:00080000:1.0:1789762332.123012:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762332.123014:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762332.123017:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762332.123024:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565763328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762332.123040:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762332.123101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762332/real 1789762332] req@ffff93f844cca300 x1876699565762944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762332.123123:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565762944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762332.123143:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762332.123146:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762332.123149:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565762944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762332.123177:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565763072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762332.123212:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762332.123226:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762332/real 1789762332] req@ffff93f844cc8380 x1876699565763072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762332.123236:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565763072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762332.123242:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762332.123244:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762332.123247:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565763072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762332.123251:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565763200:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762332.123267:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762332.123274:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762332/real 1789762332] req@ffff93f844cc8000 x1876699565763200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762332.123282:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565763200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762332.123292:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762332.123295:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762332.123297:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565763200:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762332.123300:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565763328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762332.123311:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762332.123316:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762332/real 1789762332] req@ffff93f844ccbb80 x1876699565763328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762332.123323:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565763328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762332.123329:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762332.123331:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762332.123332:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565763328:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762332.186833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762333.210846:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762334.234827:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762335.258815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762336.282807:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762337.242755:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762337.242765:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762337.242770:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762337.242781:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2272 00000100:00080000:3.0:1789762337.242793:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762337.242802:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762337.242869:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565763456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762337.242895:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762337.242899:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762337.242902:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762337.242908:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2272 00000100:00080000:3.0:1789762337.242913:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789762337.242920:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565763456:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762337.242922:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762337.242954:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762337.242977:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565763584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762337.242992:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762337.242996:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762337.242999:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762337.243005:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2272 00000100:00080000:3.0:1789762337.243020:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762337.243025:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762337.243031:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789762337.243039:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762337.243054:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565763712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762337.243070:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762337.243074:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762337.243077:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762337.243084:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2272 00000100:00080000:3.0:1789762337.243088:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762337.243093:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762337.243100:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762337.243108:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762337/real 1789762337] req@ffff93f844d64e00 x1876699565763456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762337.243120:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565763840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762337.243138:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565763456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762337.243152:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762337.243157:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762337.243164:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565763456:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762337.243215:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565763584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762337.243234:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762337.243247:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762337/real 1789762337] req@ffff93f844d65c00 x1876699565763584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762337.243260:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565763584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762337.243270:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762337.243273:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762337.243277:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565763584:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762337.243285:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565763712:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762337.243306:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762337.243314:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762337/real 1789762337] req@ffff93f844d66a00 x1876699565763712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762337.243322:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565763712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762337.243329:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762337.243331:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762337.243333:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565763712:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762337.243337:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565763840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762337.243349:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762337.243355:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762337/real 1789762337] req@ffff93f844d67100 x1876699565763840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762337.243363:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565763840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762337.243368:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762337.243370:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762337.243372:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565763840:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762337.306831:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762338.330937:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762339.354907:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762340.378886:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762341.402895:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762342.362760:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762342.362772:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762342.362777:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762342.362788:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2277 00000100:00080000:2.0:1789762342.362798:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762342.362821:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762342.362863:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565763968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762342.362889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762342.362892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762342.362894:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762342.362897:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2277 00000100:00080000:2.0:1789762342.362900:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762342.362903:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762342.362912:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762342.362922:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565764096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762342.362933:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565763968:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762342.362937:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762342.362942:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762342.362943:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762342.362947:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2277 00000100:00080000:2.0:1789762342.362949:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762342.362952:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762342.362955:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762342.362964:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565764224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762342.362972:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762342.362975:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762342.362977:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762342.362979:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2277 00000100:00080000:2.0:1789762342.362982:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762342.362985:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762342.362988:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762342.362996:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565764352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789762342.363006:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762342.363050:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762342/real 1789762342] req@ffff93f8517c0a80 x1876699565763968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762342.363065:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565763968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762342.363074:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762342.363077:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762342.363080:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565763968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762342.363103:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565764096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762342.363118:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762342.363126:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762342/real 1789762342] req@ffff93f8517c2680 x1876699565764096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762342.363134:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565764096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762342.363140:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762342.363143:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762342.363145:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565764096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762342.363149:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565764224:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762342.363160:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762342.363168:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762342/real 1789762342] req@ffff93f8517c1500 x1876699565764224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762342.363175:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565764224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762342.363181:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762342.363182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762342.363184:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565764224:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762342.363187:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565764352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762342.363205:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762342.363213:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762342/real 1789762342] req@ffff93f8517c0380 x1876699565764352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762342.363221:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565764352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762342.363227:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762342.363229:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762342.363231:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565764352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762342.426880:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762343.450807:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762344.474871:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762345.498895:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762346.522876:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762347.482836:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762347.482849:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762347.482854:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762347.482867:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2282 00000100:00080000:1.0:1789762347.482882:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762347.482894:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762347.482946:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565764480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762347.482979:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762347.482983:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762347.482985:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762347.482989:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2282 00000100:00080000:1.0:1789762347.482993:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762347.482998:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762347.483009:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762347.483022:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565764480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762347.483024:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565764608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762347.483041:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762347.483045:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762347.483046:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762347.483051:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2282 00000100:00080000:1.0:1789762347.483054:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762347.483057:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762347.483061:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762347.483072:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565764736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762347.483082:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762347.483086:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762347.483088:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762347.483091:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2282 00000100:00080000:1.0:1789762347.483096:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762347.483100:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762347.483105:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789762347.483118:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762347.483119:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565764864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762347.483193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762347/real 1789762347] req@ffff93f844cc8700 x1876699565764480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762347.483221:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565764480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762347.483233:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762347.483237:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762347.483241:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565764480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762347.483274:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565764608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762347.483296:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762347.483307:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762347/real 1789762347] req@ffff93f844ccb800 x1876699565764608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762347.483318:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565764608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762347.483326:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762347.483329:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762347.483331:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565764608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762347.483336:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565764736:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762347.483362:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762347.483371:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762347/real 1789762347] req@ffff93f844ccb480 x1876699565764736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762347.483382:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565764736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762347.483395:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762347.483398:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762347.483401:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565764736:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762347.483405:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565764864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762347.483419:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762347.483427:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762347/real 1789762347] req@ffff93f844ccaa00 x1876699565764864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762347.483447:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565764864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762347.483454:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762347.483456:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762347.483459:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565764864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762347.546956:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762348.570890:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762349.594849:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762350.618778:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762351.642795:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762352.602705:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762352.602715:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762352.602720:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762352.602744:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2287 00000100:00080000:1.0:1789762352.602755:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762352.602764:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762352.602809:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565764992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762352.602832:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762352.602836:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762352.602837:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762352.602841:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2287 00000100:00080000:1.0:1789762352.602846:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762352.602852:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762352.602864:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762352.602874:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565764992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762352.602885:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565765120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762352.602901:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762352.602906:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762352.602908:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762352.602914:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2287 00000100:00080000:1.0:1789762352.602919:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762352.602924:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762352.602931:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762352.602944:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565765248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762352.602952:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762352.602956:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762352.602959:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789762352.602959:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762352.602967:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2287 00000100:00080000:1.0:1789762352.602971:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762352.602974:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762352.602978:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762352.602986:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565765376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762352.603025:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762352/real 1789762352] req@ffff93f844ccb100 x1876699565764992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762352.603046:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565764992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762352.603056:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762352.603060:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762352.603063:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565764992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762352.603088:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565765120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762352.603111:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762352.603121:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762352/real 1789762352] req@ffff93f844cc9c00 x1876699565765120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762352.603130:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565765120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762352.603137:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762352.603140:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762352.603143:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565765120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762352.603147:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565765248:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762352.603162:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762352.603171:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762352/real 1789762352] req@ffff93f844cca680 x1876699565765248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762352.603180:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565765248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762352.603188:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762352.603191:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762352.603194:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565765248:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762352.603197:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565765376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762352.603212:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762352.603223:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762352/real 1789762352] req@ffff93f844cc8e00 x1876699565765376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762352.603234:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565765376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762352.603240:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762352.603243:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762352.603246:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565765376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762352.666803:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762353.690877:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762354.714904:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762355.739025:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762356.762804:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762357.722892:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762357.722918:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762357.722925:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762357.722942:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2292 00000100:00080000:2.0:1789762357.722958:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762357.722971:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762357.723053:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565765504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762357.723084:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762357.723089:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762357.723091:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762357.723095:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2292 00000100:00080000:2.0:1789762357.723110:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762357.723114:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762357.723126:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762357.723156:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565765632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762357.723200:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565765504:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762357.723207:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762357.723215:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762357.723219:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762357.723227:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2292 00000100:00080000:2.0:1789762357.723245:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762357.723253:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762357.723261:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762357.723281:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565765760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762357.723292:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762357.723296:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762357.723298:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1789762357.723305:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762357.723306:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2292 00000100:00080000:2.0:1789762357.723319:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762357.723336:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762357.723342:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762357.723357:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565765888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762357.723364:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762357/real 1789762357] req@ffff93f8517c1180 x1876699565765504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762357.723404:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565765504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762357.723415:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762357.723419:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762357.723422:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565765504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762357.723457:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565765632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762357.723475:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762357.723487:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762357/real 1789762357] req@ffff93f8517c3100 x1876699565765632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762357.723500:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565765632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762357.723527:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762357.723531:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762357.723535:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565765632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762357.723544:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565765760:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762357.723569:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762357.723578:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762357/real 1789762357] req@ffff93f8517c2a00 x1876699565765760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762357.723588:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565765760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762357.723599:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762357.723613:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762357.723615:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565765760:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762357.723620:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565765888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762357.723634:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762357.723641:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762357/real 1789762357] req@ffff93f8517c2d80 x1876699565765888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762357.723649:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565765888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762357.723655:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762357.723706:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762357.723711:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565765888:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762357.786862:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762358.810882:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762359.834798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762360.858831:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762361.882817:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762362.842795:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762362.842808:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762362.842814:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762362.842826:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2297 00000100:00080000:3.0:1789762362.842839:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762362.842851:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762362.842895:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565766016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762362.842934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762362.842939:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762362.842941:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762362.842946:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2297 00000100:00080000:3.0:1789762362.842952:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762362.842958:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762362.842987:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762362.843010:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565766016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762362.843010:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565766144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762362.843054:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762362.843058:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762362.843061:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762362.843067:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2297 00000100:00080000:3.0:1789762362.843073:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762362.843078:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762362.843086:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762362.843103:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565766272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762362.843133:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762362.843138:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762362.843142:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1789762362.843145:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762362.843154:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2297 00000100:00080000:3.0:1789762362.843162:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762362.843169:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762362.843175:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762362.843209:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565766400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762362.843232:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762362/real 1789762362] req@ffff93f844d67800 x1876699565766016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762362.843259:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565766016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762362.843273:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762362.843280:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762362.843286:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565766016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762362.843329:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565766144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762362.843368:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762362.843387:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762362/real 1789762362] req@ffff93f844d67480 x1876699565766144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762362.843412:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565766144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762362.843422:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762362.843426:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762362.843431:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565766144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762362.843439:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565766272:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762362.843463:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762362.843477:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762362/real 1789762362] req@ffff93f844d65880 x1876699565766272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762362.843489:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565766272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762362.843498:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762362.843501:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762362.843505:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565766272:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762362.843511:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565766400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762362.843533:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762362.843546:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762362/real 1789762362] req@ffff93f844d66680 x1876699565766400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762362.843559:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565766400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762362.843567:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762362.843571:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762362.843574:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565766400:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762362.906796:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762363.930791:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762364.954869:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762365.978783:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762367.002927:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762367.962745:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762367.962755:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762367.962760:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762367.962771:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2302 00000100:00080000:2.0:1789762367.962785:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762367.962793:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762367.962834:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565766528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762367.962865:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762367.962869:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762367.962871:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762367.962875:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2302 00000100:00080000:2.0:1789762367.962878:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762367.962881:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762367.962891:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762367.962903:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565766656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762367.962903:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565766528:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762367.962917:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762367.962919:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762367.962920:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762367.962923:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2302 00000100:00080000:2.0:1789762367.962926:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762367.962928:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762367.962932:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762367.962940:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565766784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762367.962948:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762367.962952:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762367.962953:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762367.962956:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2302 00000100:00080000:2.0:1789762367.962966:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762367.962969:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762367.962972:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762367.962981:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565766912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789762367.962989:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762367.963037:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762367/real 1789762367] req@ffff93f8517c0e00 x1876699565766528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762367.963055:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565766528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762367.963065:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762367.963068:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762367.963072:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565766528:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762367.963095:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565766656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762367.963112:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762367.963121:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762367/real 1789762367] req@ffff93f8517c3480 x1876699565766656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762367.963129:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565766656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762367.963140:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762367.963142:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762367.963145:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565766656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762367.963148:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565766784:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762367.963160:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762367.963166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762367/real 1789762367] req@ffff93f8517c3800 x1876699565766784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762367.963173:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565766784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762367.963179:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762367.963180:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762367.963182:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565766784:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762367.963185:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565766912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762367.963196:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762367.963201:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762367/real 1789762367] req@ffff93f8517c1880 x1876699565766912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762367.963209:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565766912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762367.963214:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762367.963215:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762367.963218:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565766912:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762368.026761:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762369.050814:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762370.074815:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762371.098842:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762372.122807:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762373.082760:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762373.082774:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762373.082781:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762373.082794:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2308 00000100:00080000:2.0:1789762373.082805:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762373.082815:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762373.082863:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565767040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762373.082889:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762373.082893:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762373.082895:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762373.082900:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2308 00000100:00080000:2.0:1789762373.082916:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762373.082921:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762373.082929:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565767040:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789762373.082932:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762373.082949:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565767168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762373.082962:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762373.082966:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762373.082968:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762373.082972:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2308 00000100:00080000:2.0:1789762373.082989:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762373.082993:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762373.082998:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789762373.083003:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762373.083014:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565767296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762373.083038:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762373.083042:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762373.083044:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762373.083048:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2308 00000100:00080000:2.0:1789762373.083052:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762373.083056:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762373.083062:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762373.083070:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762373/real 1789762373] req@ffff93f8517c1880 x1876699565767040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762373.083076:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565767424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762373.083108:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565767040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762373.083119:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762373.083123:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762373.083127:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565767040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762373.083150:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565767168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762373.083167:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762373.083176:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762373/real 1789762373] req@ffff93f8517c3800 x1876699565767168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762373.083184:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565767168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762373.083190:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762373.083193:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762373.083195:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565767168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762373.083199:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565767296:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762373.083212:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762373.083218:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762373/real 1789762373] req@ffff93f8517c3480 x1876699565767296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762373.083226:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565767296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762373.083232:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762373.083233:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762373.083235:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565767296:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762373.083239:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565767424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762373.083250:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762373.083256:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762373/real 1789762373] req@ffff93f8517c0e00 x1876699565767424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762373.083263:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565767424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762373.083268:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762373.083270:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762373.083272:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565767424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762373.146770:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762374.170755:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762375.194833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762376.218826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762377.242838:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762378.202818:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762378.202829:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762378.202835:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762378.202845:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2313 00000100:00080000:2.0:1789762378.202860:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762378.202869:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762378.202912:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565767552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762378.202967:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762378.202971:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762378.202973:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762378.202980:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2313 00000100:00080000:2.0:1789762378.202985:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762378.202989:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762378.202999:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762378.203012:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565767680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762378.203025:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762378.203028:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762378.203030:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762378.203037:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2313 00000100:00100000:3.0:1789762378.203039:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565767552:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762378.203040:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762378.203049:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762378.203056:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762378.203075:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565767808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762378.203089:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762378.203095:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762378.203098:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762378.203103:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2313 00000100:00080000:2.0:1789762378.203111:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762378.203117:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762378.203124:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789762378.203129:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762378.203143:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565767936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762378.203211:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762378/real 1789762378] req@ffff93f8517c2d80 x1876699565767552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762378.203246:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565767552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762378.203260:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762378.203265:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762378.203271:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565767552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762378.203311:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565767680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762378.203349:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762378.203369:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762378/real 1789762378] req@ffff93f8517c2a00 x1876699565767680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762378.203388:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565767680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762378.203401:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762378.203406:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762378.203411:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565767680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762378.203418:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565767808:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762378.203443:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762378.203464:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762378/real 1789762378] req@ffff93f8517c3100 x1876699565767808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762378.203476:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565767808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762378.203489:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762378.203492:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762378.203496:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565767808:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762378.203501:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565767936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762378.203518:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762378.203527:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762378/real 1789762378] req@ffff93f8517c1180 x1876699565767936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762378.203537:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565767936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762378.203563:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762378.203566:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762378.203569:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565767936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762378.266847:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762379.290802:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762380.314814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762381.338825:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762382.362837:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762383.322723:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762383.322734:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762383.322739:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762383.322748:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2318 00000100:00080000:3.0:1789762383.322757:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762383.322764:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762383.322802:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565768064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762383.322840:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762383.322845:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762383.322847:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762383.322851:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2318 00000100:00080000:3.0:1789762383.322854:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762383.322857:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762383.322865:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762383.322875:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565768192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762383.322882:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762383.322885:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762383.322886:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762383.322888:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2318 00000100:00080000:3.0:1789762383.322891:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762383.322893:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762383.322896:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762383.322905:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565768320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762383.322913:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762383.322915:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762383.322917:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789762383.322921:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565768064:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762383.322922:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2318 00000100:00080000:3.0:1789762383.322940:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762383.322943:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762383.322946:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762383.322956:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565768448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762383.323050:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565768192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762383.323092:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565768320:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762383.323130:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565768448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762383.386764:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762384.410763:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762385.434867:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789762386.395026:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789762386.395092:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789762386.395113:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762386.395162:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762386.395177:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762386.395190:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789762386.395223:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762383/real 1789762386] req@ffff93f844d66680 x1876699565768320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762386.395256:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565768320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762386.395284:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762386.395291:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762386.395296:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565768320:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762386.395306:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762383/real 1789762386] req@ffff93f844d65880 x1876699565768448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762386.395322:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565768448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762386.395331:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762386.395336:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762386.395340:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565768448:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762386.395348:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762383/real 1789762386] req@ffff93f844d67b80 x1876699565768064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762386.395363:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565768064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762386.395372:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762386.395377:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762386.395380:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565768064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762386.395388:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762383/real 1789762386] req@ffff93f844d64700 x1876699565768192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762386.395401:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565768192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762386.395411:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762386.395416:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762386.395419:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565768192:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762386.458865:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762387.482817:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762388.442749:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762388.442756:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762388.442760:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762388.442769:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2323 00000100:00080000:0.0:1789762388.442777:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762388.442783:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762388.442812:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565768576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762388.442835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762388.442837:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762388.442839:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762388.442841:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2323 00000100:00080000:0.0:1789762388.442844:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762388.442846:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762388.442854:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762388.442863:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565768704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762388.442871:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565768576:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762388.442873:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762388.442880:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762388.442882:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762388.442885:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2323 00000100:00080000:0.0:1789762388.442887:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762388.442889:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762388.442893:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762388.442901:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565768832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762388.442908:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762388.442912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762388.442913:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762388.442916:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2323 00000100:00080000:0.0:1789762388.442918:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762388.442920:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762388.442923:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762388.442930:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565768960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789762388.442941:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762388.442987:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762388/real 1789762388] req@ffff93f850d43100 x1876699565768576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762388.443009:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565768576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762388.443020:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762388.443023:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762388.443027:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565768576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762388.443061:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565768704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762388.443099:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762388.443111:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762388/real 1789762388] req@ffff93f850d41180 x1876699565768704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762388.443123:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565768704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762388.443131:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762388.443134:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762388.443138:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565768704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762388.443146:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565768832:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762388.443177:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762388.443205:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762388/real 1789762388] req@ffff93f850d40000 x1876699565768832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762388.443220:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565768832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762388.443230:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762388.443235:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762388.443238:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565768832:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762388.443246:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565768960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762388.443275:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762388.443291:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762388/real 1789762388] req@ffff93f850d42680 x1876699565768960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762388.443310:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565768960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762388.443318:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762388.443321:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762388.443325:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565768960:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762388.506812:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762389.530769:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762390.554819:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762391.578820:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762392.602819:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762393.562770:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762393.562781:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762393.562788:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762393.562799:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2328 00000100:00080000:0.0:1789762393.562809:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762393.562818:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762393.562862:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565769088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762393.562897:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762393.562900:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762393.562902:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762393.562905:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2328 00000100:00080000:0.0:1789762393.562907:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762393.562910:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762393.562918:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762393.562927:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565769216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762393.562934:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762393.562937:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762393.562938:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762393.562940:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2328 00000100:00080000:0.0:1789762393.562942:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762393.562944:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762393.562947:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762393.562963:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565769344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762393.562976:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762393.562979:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762393.562980:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762393.562983:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2328 00000100:00080000:0.0:1789762393.562985:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762393.562987:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762393.562990:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762393.562997:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565769472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762393.563003:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565769088:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1789762393.563088:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762393.563140:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762393/real 1789762393] req@ffff93f850d41c00 x1876699565769088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762393.563156:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565769088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762393.563165:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762393.563168:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762393.563171:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565769088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762393.563176:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565769216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762393.563193:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762393.563204:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762393/real 1789762393] req@ffff93f850d40e00 x1876699565769216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762393.563212:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565769216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762393.563222:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762393.563224:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762393.563226:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565769216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762393.563230:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565769344:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762393.563252:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762393.563259:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762393/real 1789762393] req@ffff93f850d40700 x1876699565769344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762393.563265:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565769344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762393.563271:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762393.563273:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762393.563275:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565769344:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762393.563298:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565769472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762393.563309:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762393.563315:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762393/real 1789762393] req@ffff93f850d43b80 x1876699565769472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762393.563322:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565769472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762393.563328:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762393.563329:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762393.563332:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565769472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762393.626824:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762394.650986:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762395.674807:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762396.698770:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762397.722864:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762398.682748:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762398.682759:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762398.682780:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762398.682790:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2333 00000100:00080000:0.0:1789762398.682800:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762398.682808:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762398.682843:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565769600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762398.682865:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762398.682868:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762398.682870:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762398.682872:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2333 00000100:00080000:0.0:1789762398.682876:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762398.682880:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762398.682888:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762398.682898:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565769728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762398.682905:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565769600:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762398.682908:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762398.682914:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762398.682915:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762398.682918:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2333 00000100:00080000:0.0:1789762398.682920:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762398.682922:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762398.682925:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762398.682934:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565769856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762398.682941:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762398.682944:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762398.682945:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762398.682947:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2333 00000100:00080000:0.0:1789762398.682949:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762398.682952:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762398.682955:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762398.682963:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565769984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789762398.682969:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762398.683009:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762398/real 1789762398] req@ffff93f850d40380 x1876699565769600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762398.683022:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565769600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762398.683030:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762398.683033:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762398.683037:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565769600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762398.683055:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565769728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762398.683070:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762398.683078:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762398/real 1789762398] req@ffff93f850d42300 x1876699565769728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762398.683085:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565769728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762398.683091:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762398.683092:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762398.683094:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565769728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762398.683098:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565769856:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762398.683108:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762398.683122:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762398/real 1789762398] req@ffff93f850d43b80 x1876699565769856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762398.683129:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565769856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762398.683135:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762398.683137:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762398.683139:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565769856:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762398.683142:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565769984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762398.683153:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762398.683159:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762398/real 1789762398] req@ffff93f850d40700 x1876699565769984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762398.683166:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565769984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762398.683171:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762398.683173:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762398.683174:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565769984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762398.746783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762399.770819:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762400.794864:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762401.818878:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762402.842853:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762403.802817:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762403.802827:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762403.802837:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762403.802847:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2338 00000100:00080000:2.0:1789762403.802857:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762403.802864:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762403.802900:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565770112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762403.802923:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762403.802926:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762403.802928:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762403.802931:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2338 00000100:00080000:2.0:1789762403.802934:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762403.802938:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762403.802946:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762403.802956:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565770240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762403.802964:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565770112:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762403.802968:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762403.802975:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762403.802977:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762403.802980:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2338 00000100:00080000:2.0:1789762403.802983:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762403.802985:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762403.802989:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762403.802998:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565770368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762403.803006:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762403.803009:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762403.803010:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762403.803013:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2338 00000100:00080000:2.0:1789762403.803025:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762403.803033:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762403.803037:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789762403.803042:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762403.803047:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565770496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762403.803098:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762403/real 1789762403] req@ffff93f8517c0380 x1876699565770112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762403.803118:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565770112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762403.803128:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762403.803135:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762403.803140:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565770112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762403.803162:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565770240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762403.803188:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762403.803198:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762403/real 1789762403] req@ffff93f8517c1500 x1876699565770240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762403.803207:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565770240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762403.803214:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762403.803216:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762403.803218:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565770240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762403.803222:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565770368:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762403.803235:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762403.803242:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762403/real 1789762403] req@ffff93f8517c2680 x1876699565770368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762403.803250:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565770368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762403.803261:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762403.803263:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762403.803265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565770368:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762403.803269:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565770496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762403.803280:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762403.803292:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762403/real 1789762403] req@ffff93f8517c0a80 x1876699565770496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762403.803301:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565770496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762403.803307:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762403.803309:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762403.803311:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565770496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789762403.866838:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762404.890809:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762405.914765:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762406.938846:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762407.962835:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762408.922792:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762408.922809:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762408.922814:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762408.922827:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2343 00000100:00080000:0.0:1789762408.922840:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762408.922852:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762408.922901:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565770624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762408.922937:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762408.922942:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762408.922944:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762408.922949:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2343 00000100:00080000:0.0:1789762408.922954:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762408.922960:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762408.922971:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762408.922990:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565770752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762408.922998:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565770624:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762408.923012:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762408.923017:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762408.923020:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762408.923025:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2343 00000100:00080000:0.0:1789762408.923030:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762408.923034:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762408.923041:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762408.923056:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565770880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762408.923067:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762408.923072:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762408.923075:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1789762408.923084:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762408.923086:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2343 00000100:00080000:0.0:1789762408.923097:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762408.923104:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762408.923114:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762408.923137:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565771008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762408.923164:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762408/real 1789762408] req@ffff93f850d40e00 x1876699565770624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762408.923190:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565770624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762408.923203:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762408.923209:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762408.923214:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565770624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762408.923241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565770752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762408.923270:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762408.923284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762408/real 1789762408] req@ffff93f850d41c00 x1876699565770752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762408.923297:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565770752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762408.923306:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762408.923331:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762408.923336:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565770752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762408.923344:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565770880:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762408.923390:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762408.923409:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762408/real 1789762408] req@ffff93f850d42680 x1876699565770880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762408.923442:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565770880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762408.923453:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762408.923457:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762408.923460:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565770880:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762408.923467:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565771008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762408.923491:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762408.923503:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762408/real 1789762408] req@ffff93f850d40000 x1876699565771008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762408.923515:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565771008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762408.923523:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762408.923526:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762408.923529:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565771008:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762408.986811:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762410.010772:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762411.034784:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762412.058855:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762413.082790:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762414.042767:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762414.042775:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762414.042779:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762414.042794:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2349 00000100:00080000:2.0:1789762414.042805:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762414.042811:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762414.042844:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565771136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762414.042867:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762414.042869:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762414.042871:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762414.042873:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2349 00000100:00080000:2.0:1789762414.042876:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762414.042879:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762414.042887:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762414.042896:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565771264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762414.042904:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762414.042910:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:3.0:1789762414.042912:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565771136:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762414.042913:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762414.042931:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2349 00000100:00080000:2.0:1789762414.042934:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762414.042937:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762414.042940:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762414.042950:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565771392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762414.042957:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762414.042960:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762414.042961:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762414.042964:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2349 00000100:00080000:2.0:1789762414.042967:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762414.042970:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762414.042972:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762414.042979:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565771520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789762414.043003:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762414.043055:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762414/real 1789762414] req@ffff93f8517c2300 x1876699565771136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762414.043080:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565771136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762414.043089:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762414.043093:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762414.043096:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565771136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762414.043121:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565771264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762414.043140:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762414.043149:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762414/real 1789762414] req@ffff93f8517c1f80 x1876699565771264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762414.043158:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565771264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762414.043164:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762414.043166:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762414.043168:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565771264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762414.043172:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565771392:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762414.043184:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762414.043190:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762414/real 1789762414] req@ffff93f8517c3b80 x1876699565771392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762414.043198:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565771392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762414.043203:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762414.043205:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762414.043207:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565771392:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762414.043217:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565771520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762414.043230:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762414.043237:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762414/real 1789762414] req@ffff93f8517c0000 x1876699565771520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762414.043249:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565771520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762414.043255:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762414.043257:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762414.043258:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565771520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762414.106812:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762415.130792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762416.154804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762417.178798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762418.202770:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762419.162755:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762419.162763:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762419.162766:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762419.162774:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2354 00000100:00080000:2.0:1789762419.162783:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762419.162789:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762419.162823:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565771648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762419.162849:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762419.162853:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762419.162855:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762419.162859:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2354 00000100:00080000:2.0:1789762419.162864:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762419.162870:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762419.162882:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762419.162901:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565771776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762419.162903:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565771648:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762419.162927:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762419.162929:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762419.162931:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762419.162935:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2354 00000100:00080000:2.0:1789762419.162939:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762419.162943:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762419.162947:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762419.162956:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565771904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762419.162973:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762419.162976:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762419.162977:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762419.162980:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2354 00000100:00080000:2.0:1789762419.162983:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762419.162986:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:3.0:1789762419.162993:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1789762419.162996:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762419.163008:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565772032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762419.163061:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762419/real 1789762419] req@ffff93f8517c0000 x1876699565771648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762419.163084:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565771648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762419.163099:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762419.163105:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762419.163110:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565771648:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762419.163142:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565771776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762419.163169:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762419.163192:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762419/real 1789762419] req@ffff93f8517c3b80 x1876699565771776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762419.163209:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565771776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762419.163216:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762419.163218:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762419.163220:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565771776:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762419.163225:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565771904:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762419.163258:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762419.163267:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762419/real 1789762419] req@ffff93f8517c1f80 x1876699565771904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762419.163276:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565771904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762419.163282:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762419.163284:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762419.163287:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565771904:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762419.163291:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565772032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762419.163304:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762419.163310:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762419/real 1789762419] req@ffff93f8517c2300 x1876699565772032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762419.163317:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565772032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762419.163330:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762419.163333:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762419.163337:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565772032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762419.226835:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762420.250791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762421.274764:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762422.298807:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762423.322848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762424.282801:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762424.282813:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762424.282820:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762424.282834:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2359 00000100:00080000:2.0:1789762424.282857:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762424.282865:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762424.282905:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565772160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762424.282939:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762424.282945:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762424.282948:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762424.282957:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2359 00000100:00080000:2.0:1789762424.282965:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762424.282972:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762424.282987:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762424.282995:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565772160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762424.283007:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565772288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762424.283034:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762424.283040:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762424.283043:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762424.283049:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2359 00000100:00080000:2.0:1789762424.283055:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762424.283060:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762424.283068:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789762424.283077:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762424.283084:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565772416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762424.283098:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762424.283103:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762424.283107:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762424.283111:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2359 00000100:00080000:2.0:1789762424.283116:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762424.283121:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762424.283128:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762424/real 1789762424] req@ffff93f8517c0a80 x1876699565772160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1789762424.283128:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762424.283141:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565772544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762424.283163:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565772160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762424.283177:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762424.283184:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762424.283193:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565772160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762424.283249:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565772288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762424.283282:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762424.283297:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762424/real 1789762424] req@ffff93f8517c2680 x1876699565772288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762424.283313:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565772288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762424.283322:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762424.283326:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762424.283330:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565772288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762424.283336:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565772416:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762424.283361:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762424.283380:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762424/real 1789762424] req@ffff93f8517c1500 x1876699565772416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762424.283399:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565772416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762424.283428:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762424.283433:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762424.283438:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565772416:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762424.283447:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565772544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762424.283483:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762424.283501:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762424/real 1789762424] req@ffff93f8517c0380 x1876699565772544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762424.283529:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565772544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762424.283542:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762424.283547:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762424.283552:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565772544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762424.346784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762425.370906:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762426.394936:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762427.418858:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762428.442823:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762429.402884:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762429.402896:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762429.402901:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762429.402914:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2364 00000100:00080000:0.0:1789762429.402926:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762429.402934:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762429.402973:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565772672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762429.403007:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762429.403012:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762429.403014:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762429.403018:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2364 00000100:00080000:0.0:1789762429.403021:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762429.403025:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762429.403034:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762429.403051:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565772800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762429.403052:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565772672:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762429.403092:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762429.403097:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762429.403100:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762429.403105:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2364 00000100:00080000:0.0:1789762429.403110:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762429.403114:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762429.403121:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789762429.403126:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762429.403138:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565772928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762429.403151:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762429.403154:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762429.403155:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762429.403173:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2364 00000100:00080000:0.0:1789762429.403177:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762429.403183:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762429.403184:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762429/real 1789762429] req@ffff93f850d41180 x1876699565772672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1789762429.403187:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762429.403206:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565773056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762429.403212:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565772672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762429.403234:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762429.403238:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762429.403243:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565772672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762429.403263:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565772800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762429.403282:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762429.403292:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762429/real 1789762429] req@ffff93f850d43100 x1876699565772800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762429.403327:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565772800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762429.403336:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762429.403339:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762429.403343:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565772800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762429.403350:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565772928:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762429.403372:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762429.403384:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762429/real 1789762429] req@ffff93f850d41f80 x1876699565772928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762429.403405:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565772928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762429.403411:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762429.403413:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762429.403415:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565772928:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762429.403420:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565773056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762429.403435:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762429.403443:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762429/real 1789762429] req@ffff93f850d41500 x1876699565773056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762429.403450:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565773056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762429.403464:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762429.403466:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762429.403468:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565773056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762429.466844:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762430.490841:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762431.514798:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762432.538792:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762433.562840:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762434.522821:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762434.522837:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762434.522845:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762434.522862:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2369 00000100:00080000:0.0:1789762434.522878:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762434.522888:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762434.522937:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565773184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762434.522972:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762434.522976:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762434.522979:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762434.522984:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2369 00000100:00080000:0.0:1789762434.522988:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762434.522993:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762434.523005:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762434.523020:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565773312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762434.523035:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762434.523039:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762434.523046:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1789762434.523048:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565773184:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762434.523054:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2369 00000100:00080000:0.0:1789762434.523067:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762434.523074:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762434.523084:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762434.523113:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565773440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762434.523158:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1789762434.523167:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762434.523174:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762434.523181:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762434.523191:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2369 00000100:00080000:0.0:1789762434.523199:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762434.523205:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762434.523214:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762434.523226:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762434/real 1789762434] req@ffff93f850d43480 x1876699565773184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762434.523242:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565773568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762434.523258:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565773184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762434.523284:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762434.523290:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762434.523296:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565773184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762434.523327:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565773312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762434.523355:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762434.523369:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762434/real 1789762434] req@ffff93f850d43800 x1876699565773312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762434.523384:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565773312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762434.523402:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762434.523406:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762434.523410:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565773312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762434.523416:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565773440:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762434.523436:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762434.523447:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762434/real 1789762434] req@ffff93f850d41500 x1876699565773440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762434.523460:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565773440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762434.523470:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762434.523473:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762434.523476:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565773440:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762434.523482:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565773568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762434.523500:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762434.523511:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762434/real 1789762434] req@ffff93f850d41f80 x1876699565773568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762434.523525:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565773568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762434.523533:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762434.523537:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762434.523541:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565773568:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762434.586839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762435.610894:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762436.634891:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762437.658811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762438.682823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762439.642844:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762439.642858:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762439.642864:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762439.642877:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2374 00000100:00080000:0.0:1789762439.642894:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762439.642908:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762439.642969:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565773696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762439.643013:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762439.643018:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762439.643021:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762439.643027:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2374 00000100:00080000:0.0:1789762439.643034:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762439.643040:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762439.643052:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762439.643117:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565773824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762439.643129:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565773696:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762439.643153:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762439.643161:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762439.643165:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762439.643174:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2374 00000100:00080000:0.0:1789762439.643200:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762439.643209:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762439.643222:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762439.643251:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565773952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789762439.643261:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762439.643287:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762439.643294:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762439.643299:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762439.643309:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2374 00000100:00080000:0.0:1789762439.643318:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762439.643325:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762439.643335:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762439.643358:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565774080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762439.643384:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762439/real 1789762439] req@ffff93f850d43100 x1876699565773696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762439.643440:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565773696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762439.643465:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762439.643474:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762439.643482:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565773696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762439.643520:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565773824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762439.643555:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762439.643573:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762439/real 1789762439] req@ffff93f850d41180 x1876699565773824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762439.643588:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565773824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762439.643599:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762439.643603:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762439.643607:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565773824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762439.643617:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565773952:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762439.643638:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762439.643649:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762439/real 1789762439] req@ffff93f850d40000 x1876699565773952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762439.643730:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565773952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762439.643748:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762439.643755:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762439.643761:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565773952:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762439.643773:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565774080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762439.643804:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762439.643819:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762439/real 1789762439] req@ffff93f850d42680 x1876699565774080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762439.643833:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565774080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762439.643842:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762439.643846:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762439.643850:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565774080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762439.706849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762440.730857:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762441.754831:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762442.778896:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762443.802836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762444.762721:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762444.762728:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762444.762732:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762444.762740:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2379 00000100:00080000:0.0:1789762444.762749:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762444.762754:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762444.762785:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565774208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762444.762808:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762444.762810:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762444.762812:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762444.762814:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2379 00000100:00080000:0.0:1789762444.762816:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762444.762819:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762444.762826:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762444.762833:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565774336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762444.762840:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762444.762842:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762444.762843:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762444.762845:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2379 00000100:00080000:0.0:1789762444.762847:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762444.762849:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762444.762851:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762444.762870:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565774464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762444.762876:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762444.762879:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762444.762881:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762444.762883:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2379 00000100:00080000:0.0:1789762444.762885:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762444.762888:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762444.762891:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762444.762894:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565774208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762444.762899:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565774592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789762444.762967:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762444.763100:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762444/real 1789762444] req@ffff93f850d41c00 x1876699565774208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762444.763114:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565774208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762444.763133:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762444.763136:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762444.763139:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565774208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762444.763144:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565774336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762444.763182:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762444.763193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762444/real 1789762444] req@ffff93f850d40e00 x1876699565774336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762444.763203:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565774336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762444.763232:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762444.763235:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762444.763237:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565774336:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762444.763241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565774464:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762444.763254:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762444.763261:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762444/real 1789762444] req@ffff93f850d40700 x1876699565774464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762444.763275:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565774464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762444.763298:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762444.763300:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762444.763303:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565774464:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762444.763325:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565774592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762444.763340:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762444.763348:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762444/real 1789762444] req@ffff93f850d43b80 x1876699565774592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762444.763358:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565774592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762444.763364:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762444.763367:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762444.763369:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565774592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762444.826962:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762445.850879:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762446.874818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762447.898797:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762448.922922:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762449.882860:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762449.882873:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762449.882880:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762449.882893:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2384 00000100:00080000:0.0:1789762449.882906:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762449.882916:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762449.882979:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565774720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762449.883037:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762449.883061:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762449.883065:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762449.883076:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2384 00000100:00080000:0.0:1789762449.883085:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762449.883094:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762449.883103:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565774720:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1789762449.883115:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762449.883148:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565774848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762449.883174:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762449.883181:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762449.883184:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762449.883193:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2384 00000100:00080000:0.0:1789762449.883201:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762449.883208:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762449.883220:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762449.883278:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565774848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762449.883288:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565774976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762449.883313:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762449.883319:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762449.883323:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762449.883330:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2384 00000100:00080000:0.0:1789762449.883338:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1789762449.883339:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565774976:192.168.203.147@tcp:8:kworker.0 00000100:00080000:0.0:1789762449.883353:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762449.883366:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762449.883391:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565775104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762449.883464:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565775104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762449.946747:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762450.971018:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762451.994914:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789762452.955335:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789762452.955407:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789762452.955434:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789762452.955528:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789762452.955556:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789762452.955569:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789762452.956753:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762449/real 1789762452] req@ffff93f850d40700 x1876699565775104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762452.956801:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565775104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762452.956824:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762452.956833:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762452.956842:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565775104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762452.956857:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762449/real 1789762452] req@ffff93f850d43b80 x1876699565774976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762452.956885:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565774976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762452.956900:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762452.956908:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762452.956915:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565774976:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762452.956928:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762449/real 1789762452] req@ffff93f850d40380 x1876699565774848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762452.956966:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565774848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762452.957002:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762452.957009:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762452.957015:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565774848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762452.957027:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762449/real 1789762452] req@ffff93f850d42300 x1876699565774720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762452.957050:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565774720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762452.957065:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762452.957071:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762452.957080:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565774720:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789762453.018838:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762454.042835:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762455.002786:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762455.002794:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762455.002797:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762455.002806:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2389 00000100:00080000:1.0:1789762455.002813:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762455.002820:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762455.002871:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565775232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762455.002922:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762455.002925:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762455.002927:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762455.002930:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2389 00000100:00080000:1.0:1789762455.002932:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762455.002936:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762455.002944:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762455.002955:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565775360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762455.002955:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565775232:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762455.003003:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762455.003009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762455.003011:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762455.003016:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2389 00000100:00080000:1.0:1789762455.003021:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762455.003025:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762455.003031:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762455.003047:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565775488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789762455.003058:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762455.003061:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762455.003067:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762455.003069:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762455.003076:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2389 00000100:00080000:1.0:1789762455.003081:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762455.003085:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762455.003091:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762455.003103:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565775616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762455.003120:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762455/real 1789762455] req@ffff93f844cc9500 x1876699565775232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762455.003142:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565775232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762455.003153:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762455.003158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762455.003162:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565775232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762455.003186:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565775360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762455.003215:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762455.003229:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762455/real 1789762455] req@ffff93f844ccad80 x1876699565775360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762455.003241:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565775360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762455.003250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762455.003254:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762455.003258:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565775360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762455.003265:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565775488:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762455.003287:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762455.003299:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762455/real 1789762455] req@ffff93f844cc8e00 x1876699565775488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762455.003312:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565775488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762455.003320:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762455.003323:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762455.003327:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565775488:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762455.003333:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565775616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762455.003354:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762455.003366:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762455/real 1789762455] req@ffff93f844cca680 x1876699565775616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762455.003378:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565775616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762455.003387:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762455.003390:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762455.003394:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565775616:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762455.066770:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762456.090774:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762457.114867:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762458.138777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762459.162931:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762460.122796:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762460.122807:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762460.122814:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762460.122827:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2395 00000100:00080000:1.0:1789762460.122842:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762460.122853:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762460.122906:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565775744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762460.122963:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762460.122987:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762460.122991:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762460.122998:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2395 00000100:00080000:1.0:1789762460.123003:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762460.123011:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762460.123023:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565775744:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762460.123029:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762460.123072:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565775872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762460.123090:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762460.123095:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762460.123099:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762460.123103:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2395 00000100:00080000:1.0:1789762460.123118:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762460.123128:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789762460.123136:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789762460.123137:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762460.123154:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565776000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762460.123167:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762460.123171:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762460.123173:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762460.123179:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2395 00000100:00080000:1.0:1789762460.123183:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762460.123188:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762460.123197:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762460.123212:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565776128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762460.123225:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762460/real 1789762460] req@ffff93f844cc9c00 x1876699565775744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762460.123262:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565775744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762460.123274:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762460.123278:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762460.123282:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565775744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762460.123310:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565775872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762460.123344:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762460.123355:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762460/real 1789762460] req@ffff93f844ccb100 x1876699565775872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762460.123363:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565775872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762460.123370:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762460.123373:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762460.123376:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565775872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762460.123380:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565776000:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762460.123394:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762460.123403:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762460/real 1789762460] req@ffff93f844ccaa00 x1876699565776000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762460.123412:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565776000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762460.123418:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762460.123420:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762460.123422:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565776000:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762460.123425:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565776128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762460.123440:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762460.123446:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762460/real 1789762460] req@ffff93f844ccb480 x1876699565776128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762460.123455:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565776128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762460.123460:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762460.123462:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762460.123464:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565776128:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789762460.186774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762461.210861:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762462.234842:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762463.258771:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762464.282916:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762465.242792:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762465.242806:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762465.242811:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762465.242820:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2400 00000100:00080000:2.0:1789762465.242827:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762465.242834:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762465.242874:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565776256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762465.242928:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762465.242931:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762465.242933:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762465.242937:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2400 00000100:00080000:2.0:1789762465.242940:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762465.242946:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762465.242958:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762465.242977:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565776384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762465.242987:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565776256:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762465.243009:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762465.243014:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762465.243016:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762465.243021:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2400 00000100:00080000:2.0:1789762465.243027:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762465.243031:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762465.243037:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762465.243048:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565776512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762465.243057:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762465.243060:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762465.243063:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789762465.243065:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762465.243069:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2400 00000100:00080000:2.0:1789762465.243073:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762465.243076:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762465.243081:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762465.243091:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565776640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762465.243124:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762465/real 1789762465] req@ffff93f8517c1180 x1876699565776256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762465.243162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565776256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762465.243176:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762465.243182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762465.243187:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565776256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762465.243213:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565776384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762465.243236:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762465.243247:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762465/real 1789762465] req@ffff93f8517c3100 x1876699565776384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762465.243257:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565776384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762465.243264:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762465.243267:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762465.243270:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565776384:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762465.243274:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565776512:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762465.243296:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762465.243304:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762465/real 1789762465] req@ffff93f8517c2a00 x1876699565776512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762465.243312:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565776512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762465.243319:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762465.243321:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762465.243323:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565776512:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762465.243328:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565776640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762465.243340:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762465.243347:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762465/real 1789762465] req@ffff93f8517c2d80 x1876699565776640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762465.243360:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565776640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762465.243370:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762465.243374:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762465.243378:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565776640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762465.306839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762466.330817:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762467.354903:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762468.378886:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762469.402880:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762470.362767:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762470.362775:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762470.362779:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762470.362787:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2405 00000100:00080000:1.0:1789762470.362795:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762470.362801:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762470.362835:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565776768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762470.362871:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762470.362874:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762470.362876:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762470.362879:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2405 00000100:00080000:1.0:1789762470.362883:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762470.362887:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762470.362898:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762470.362909:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565776768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762470.362918:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565776896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762470.362932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762470.362934:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762470.362935:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762470.362938:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2405 00000100:00080000:1.0:1789762470.362942:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762470.362945:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762470.362949:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762470.362958:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565777024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762470.362966:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762470.362968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762470.362969:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762470.362971:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2405 00000100:00080000:1.0:1789762470.362973:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762470.362978:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789762470.362979:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789762470.362982:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762470.362990:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565777152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762470.363024:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762470/real 1789762470] req@ffff93f844ccb800 x1876699565776768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762470.363039:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565776768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762470.363048:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762470.363051:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762470.363054:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565776768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762470.363073:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565776896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762470.363088:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762470.363097:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762470/real 1789762470] req@ffff93f844cc8700 x1876699565776896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762470.363106:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565776896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762470.363112:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762470.363114:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762470.363118:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565776896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762470.363122:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565777024:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762470.363134:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762470.363142:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762470/real 1789762470] req@ffff93f844ccbb80 x1876699565777024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762470.363150:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565777024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762470.363155:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762470.363157:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762470.363159:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565777024:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762470.363162:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565777152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762470.363171:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762470.363177:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762470/real 1789762470] req@ffff93f844cc8000 x1876699565777152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762470.363184:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565777152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762470.363194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762470.363196:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762470.363198:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565777152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762470.426805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762471.450801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762472.474824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762473.498860:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762474.522886:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762475.482766:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762475.482775:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762475.482780:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762475.482789:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2410 00000100:00080000:1.0:1789762475.482800:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762475.482807:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762475.482873:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565777280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762475.482924:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762475.482927:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762475.482929:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762475.482932:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2410 00000100:00080000:1.0:1789762475.482935:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762475.482939:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762475.482948:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762475.482961:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565777408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762475.482974:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565777280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762475.482974:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762475.482983:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762475.482986:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762475.482992:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2410 00000100:00080000:1.0:1789762475.482994:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762475.482996:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762475.483000:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762475.483009:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565777536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762475.483023:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762475.483026:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762475.483028:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762475.483032:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2410 00000100:00080000:1.0:1789762475.483047:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762475.483050:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762475.483054:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789762475.483062:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762475.483066:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565777664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762475.483143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762475/real 1789762475] req@ffff93f844cc8380 x1876699565777280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762475.483167:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565777280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762475.483180:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762475.483186:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762475.483191:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565777280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762475.483217:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565777408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762475.483249:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762475.483261:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762475/real 1789762475] req@ffff93f844cca300 x1876699565777408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762475.483271:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565777408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762475.483278:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762475.483280:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762475.483283:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565777408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762475.483288:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565777536:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762475.483302:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762475.483310:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762475/real 1789762475] req@ffff93f844cc9f80 x1876699565777536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762475.483318:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565777536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762475.483324:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762475.483326:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762475.483329:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565777536:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762475.483333:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565777664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762475.483345:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762475.483352:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762475/real 1789762475] req@ffff93f844cc9880 x1876699565777664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762475.483360:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565777664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762475.483366:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762475.483368:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762475.483370:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565777664:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762475.546767:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762476.570802:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762477.594789:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762478.618777:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762479.642834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762480.602743:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762480.602753:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762480.602757:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762480.602767:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2415 00000100:00080000:2.0:1789762480.602795:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762480.602803:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762480.602848:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565777792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762480.602875:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762480.602878:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762480.602880:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762480.602884:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2415 00000100:00080000:2.0:1789762480.602888:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762480.602893:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762480.602902:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565777792:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789762480.602903:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762480.602920:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565777920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762480.602932:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762480.602935:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762480.602937:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762480.602942:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2415 00000100:00080000:2.0:1789762480.602946:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762480.602950:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762480.602955:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789762480.602962:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762480.602969:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565778048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762480.602980:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762480.602984:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762480.602985:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762480.602989:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2415 00000100:00080000:2.0:1789762480.602993:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762480.602997:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762480.603002:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762480/real 1789762480] req@ffff93f8517c0e00 x1876699565777792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1789762480.603003:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762480.603019:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565778176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762480.603023:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565777792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762480.603048:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762480.603051:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762480.603054:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565777792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762480.603074:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565777920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762480.603088:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762480.603097:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762480/real 1789762480] req@ffff93f8517c3480 x1876699565777920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762480.603105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565777920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762480.603111:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762480.603113:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762480.603115:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565777920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762480.603118:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565778048:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762480.603129:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762480.603134:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762480/real 1789762480] req@ffff93f8517c3800 x1876699565778048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762480.603142:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565778048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762480.603151:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762480.603153:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762480.603155:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565778048:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762480.603158:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565778176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762480.603168:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762480.603174:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762480/real 1789762480] req@ffff93f8517c1880 x1876699565778176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762480.603181:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565778176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762480.603186:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762480.603188:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762480.603190:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565778176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762480.666796:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762481.690812:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762482.714753:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762483.738795:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762484.762783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762485.722711:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762485.722722:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762485.722727:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762485.722746:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2420 00000100:00080000:2.0:1789762485.722756:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762485.722763:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762485.722802:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565778304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762485.722824:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762485.722828:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762485.722830:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762485.722834:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2420 00000100:00080000:2.0:1789762485.722839:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762485.722843:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762485.722852:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762485.722855:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565778304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762485.722866:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565778432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762485.722876:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762485.722879:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762485.722881:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762485.722884:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2420 00000100:00080000:2.0:1789762485.722887:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762485.722890:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762485.722894:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762485.722903:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565778560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762485.722913:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762485.722913:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762485.722916:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762485.722918:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762485.722921:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2420 00000100:00080000:2.0:1789762485.722924:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762485.722928:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762485.722932:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762485.722940:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565778688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762485.722958:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762485/real 1789762485] req@ffff93f8517c1880 x1876699565778304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762485.722979:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565778304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762485.722989:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762485.722992:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762485.722995:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565778304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762485.723015:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565778432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762485.723033:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762485.723043:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762485/real 1789762485] req@ffff93f8517c3800 x1876699565778432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762485.723051:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565778432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762485.723057:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762485.723059:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762485.723061:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565778432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762485.723065:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565778560:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762485.723075:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762485.723081:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762485/real 1789762485] req@ffff93f8517c3480 x1876699565778560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762485.723089:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565778560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762485.723094:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762485.723095:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762485.723097:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565778560:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762485.723100:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565778688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762485.723110:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762485.723119:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762485/real 1789762485] req@ffff93f8517c0e00 x1876699565778688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762485.723131:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565778688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762485.723139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762485.723143:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762485.723147:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565778688:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762485.786819:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762486.810801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762487.834842:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762488.858812:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762489.882850:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762490.842767:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762490.842775:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762490.842778:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762490.842787:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2425 00000100:00080000:0.0:1789762490.842796:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762490.842802:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762490.842834:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565778816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762490.842858:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762490.842860:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762490.842862:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762490.842865:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2425 00000100:00080000:0.0:1789762490.842867:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762490.842870:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762490.842877:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762490.842887:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565778944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762490.842896:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565778816:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762490.842904:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762490.842908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762490.842909:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762490.842913:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2425 00000100:00080000:0.0:1789762490.842915:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762490.842917:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762490.842921:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762490.842928:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565779072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762490.842935:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762490.842938:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762490.842940:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762490.842942:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2425 00000100:00080000:0.0:1789762490.842951:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762490.842954:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762490.842957:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789762490.842964:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762490.842966:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565779200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762490.843013:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762490/real 1789762490] req@ffff93f850d40e00 x1876699565778816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762490.843033:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565778816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762490.843045:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762490.843049:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762490.843053:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565778816:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762490.843077:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565778944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762490.843099:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762490.843110:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762490/real 1789762490] req@ffff93f850d41c00 x1876699565778944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762490.843124:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565778944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762490.843133:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762490.843136:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762490.843140:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565778944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762490.843147:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565779072:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762490.843166:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762490.843177:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762490/real 1789762490] req@ffff93f850d42680 x1876699565779072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762490.843190:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565779072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762490.843198:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762490.843201:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762490.843205:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565779072:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762490.843211:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565779200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762490.843227:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762490.843237:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762490/real 1789762490] req@ffff93f850d40000 x1876699565779200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762490.843248:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565779200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762490.843264:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762490.843267:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762490.843269:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565779200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762490.906817:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762491.930790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762492.954819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762493.978791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762495.002788:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762495.962772:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762495.962787:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762495.962792:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762495.962806:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2430 00000100:00080000:0.0:1789762495.962820:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762495.962829:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762495.962877:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565779328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762495.962907:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762495.962911:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762495.962913:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762495.962920:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2430 00000100:00080000:0.0:1789762495.962924:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762495.962930:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762495.962940:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762495.962954:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565779456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762495.962964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762495.962968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762495.962971:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762495.962976:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2430 00000100:00080000:0.0:1789762495.962980:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762495.962985:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762495.962990:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762495.963003:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565779584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762495.963014:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762495.963018:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762495.963020:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762495.963023:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2430 00000100:00080000:0.0:1789762495.963027:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762495.963031:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762495.963037:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762495.963050:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565779712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762495.963261:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565779328:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1789762495.963322:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762495.963361:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762495/real 1789762495] req@ffff93f850d41180 x1876699565779328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762495.963384:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565779328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762495.963393:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762495.963396:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762495.963399:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565779328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762495.963405:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565779456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762495.963439:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762495.963459:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762495/real 1789762495] req@ffff93f850d43100 x1876699565779456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762495.963473:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565779456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762495.963482:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762495.963493:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762495.963499:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565779456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762495.963505:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565779584:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762495.963523:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762495.963532:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762495/real 1789762495] req@ffff93f850d41f80 x1876699565779584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762495.963540:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565779584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762495.963546:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762495.963548:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762495.963550:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565779584:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762495.963554:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565779712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762495.963564:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762495.963571:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762495/real 1789762495] req@ffff93f850d41500 x1876699565779712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762495.963578:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565779712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762495.963583:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762495.963586:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762495.963589:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565779712:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762496.026792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762497.050799:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762498.074814:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762499.098792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762500.122795:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762501.082756:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762501.082764:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762501.082768:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762501.082777:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2436 00000100:00080000:1.0:1789762501.082785:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762501.082791:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762501.082826:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565779840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762501.082849:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762501.082851:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762501.082853:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762501.082856:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2436 00000100:00080000:1.0:1789762501.082858:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762501.082860:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762501.082868:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762501.082876:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565779968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762501.082883:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762501.082885:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762501.082886:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762501.082889:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2436 00000100:00080000:1.0:1789762501.082891:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762501.082895:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762501.082898:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565779840:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762501.082899:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762501.082911:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565780096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762501.082925:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762501.082929:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762501.082931:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762501.082935:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2436 00000100:00080000:1.0:1789762501.082939:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762501.082943:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762501.082948:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762501.082961:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565780224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762501.082972:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762501.083024:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762501/real 1789762501] req@ffff93f844cc9180 x1876699565779840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762501.083044:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565779840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762501.083054:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762501.083058:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762501.083062:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565779840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762501.083071:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565779968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762501.083095:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762501.083119:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762501/real 1789762501] req@ffff93f844cc8a80 x1876699565779968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762501.083131:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565779968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762501.083140:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762501.083143:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762501.083147:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565779968:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762501.083173:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565780096:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762501.083194:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762501.083207:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762501/real 1789762501] req@ffff93f844cc9880 x1876699565780096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762501.083219:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565780096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762501.083237:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762501.083241:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762501.083244:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565780096:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762501.083251:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565780224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762501.083270:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762501.083294:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762501/real 1789762501] req@ffff93f844cc9f80 x1876699565780224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762501.083306:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565780224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762501.083315:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762501.083317:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762501.083322:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565780224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762501.146838:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762502.170783:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762503.194819:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762504.218765:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762505.242761:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762506.202720:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762506.202728:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762506.202731:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762506.202739:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2441 00000100:00080000:1.0:1789762506.202747:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762506.202753:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762506.202786:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565780352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762506.202808:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762506.202810:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762506.202812:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762506.202814:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2441 00000100:00080000:1.0:1789762506.202816:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762506.202819:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762506.202828:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762506.202838:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565780480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762506.202845:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565780352:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762506.202849:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762506.202855:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762506.202857:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762506.202860:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2441 00000100:00080000:1.0:1789762506.202862:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762506.202864:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762506.202867:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762506.202876:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565780608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762506.202883:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762506.202886:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762506.202887:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762506.202889:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2441 00000100:00080000:1.0:1789762506.202906:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762506.202909:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762506.202912:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789762506.202918:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762506.202920:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565780736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762506.202968:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762506/real 1789762506] req@ffff93f844cca300 x1876699565780352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762506.202987:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565780352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762506.202997:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762506.203001:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762506.203005:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565780352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762506.203027:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565780480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762506.203049:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762506.203061:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762506/real 1789762506] req@ffff93f844cc8380 x1876699565780480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762506.203073:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565780480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762506.203083:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762506.203086:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762506.203090:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565780480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762506.203096:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565780608:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762506.203114:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762506.203125:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762506/real 1789762506] req@ffff93f844cc8000 x1876699565780608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762506.203136:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565780608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762506.203143:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762506.203146:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762506.203149:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565780608:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762506.203155:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565780736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762506.203172:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762506.203184:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762506/real 1789762506] req@ffff93f844ccbb80 x1876699565780736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762506.203195:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565780736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762506.203203:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762506.203206:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762506.203210:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565780736:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762506.266761:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762507.290769:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762508.314766:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762509.338800:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762510.362780:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762511.322727:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762511.322736:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762511.322742:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762511.322752:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2446 00000100:00080000:1.0:1789762511.322762:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762511.322770:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762511.322808:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565780864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762511.322832:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762511.322835:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762511.322838:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762511.322843:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2446 00000100:00080000:1.0:1789762511.322847:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762511.322851:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762511.322859:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565780864:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762511.322861:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762511.322877:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565780992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762511.322888:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762511.322892:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762511.322894:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762511.322899:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2446 00000100:00080000:1.0:1789762511.322911:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762511.322915:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789762511.322920:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789762511.322921:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762511.322935:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565781120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762511.322953:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762511.322956:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762511.322959:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762511.322963:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2446 00000100:00100000:0.0:1789762511.322968:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762511/real 1789762511] req@ffff93f844cc8700 x1876699565780864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762511.322968:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762511.322977:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762511.322986:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762511.322991:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565780864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762511.323001:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1789762511.323001:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565781248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762511.323009:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762511.323017:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565780864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762511.323045:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565780992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762511.323079:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762511.323093:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762511/real 1789762511] req@ffff93f844ccb800 x1876699565780992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762511.323105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565780992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762511.323111:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762511.323113:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762511.323115:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565780992:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762511.323120:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565781120:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762511.323136:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762511.323143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762511/real 1789762511] req@ffff93f844ccb480 x1876699565781120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762511.323150:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565781120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762511.323156:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762511.323158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762511.323160:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565781120:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762511.323164:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565781248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762511.323174:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762511.323180:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762511/real 1789762511] req@ffff93f844ccaa00 x1876699565781248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762511.323188:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565781248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762511.323192:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762511.323194:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762511.323196:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565781248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762511.386777:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762512.410774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762513.434808:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762514.458812:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762515.482851:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762516.442722:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762516.442730:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762516.442734:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762516.442742:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2451 00000100:00080000:1.0:1789762516.442750:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762516.442755:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762516.442788:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565781376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762516.442809:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762516.442812:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762516.442813:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762516.442816:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2451 00000100:00080000:1.0:1789762516.442818:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762516.442821:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762516.442829:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762516.442839:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565781504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762516.442847:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762516.442850:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762516.442851:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789762516.442857:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565781376:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762516.442857:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2451 00000100:00080000:1.0:1789762516.442860:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762516.442862:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762516.442866:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762516.442875:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565781632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762516.442883:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762516.442886:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762516.442887:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762516.442890:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2451 00000100:00080000:1.0:1789762516.442892:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762516.442894:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762516.442897:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762516.442905:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565781760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762516.442969:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565781504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762516.442992:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565781632:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762516.443009:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565781760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762516.506770:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762517.530781:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762518.554760:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789762519.514871:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789762519.514907:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789762519.514920:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762519.514951:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762519.514958:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762519.514968:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789762519.514992:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762516/real 1789762519] req@ffff93f844cc9c00 x1876699565781504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762519.515017:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565781504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762519.515029:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762519.515034:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762519.515039:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565781504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762519.515047:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762516/real 1789762519] req@ffff93f844cca680 x1876699565781632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762519.515060:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565781632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762519.515069:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762519.515073:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762519.515076:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565781632:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762519.515092:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762516/real 1789762519] req@ffff93f844cc8e00 x1876699565781760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762519.515104:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565781760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762519.515117:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762519.515121:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762519.515125:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565781760:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762519.515132:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762516/real 1789762519] req@ffff93f844ccb100 x1876699565781376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762519.515145:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565781376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762519.515154:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762519.515157:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762519.515160:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565781376:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789762519.578817:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762520.602803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762521.562824:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762521.562839:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762521.562844:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762521.562856:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2456 00000100:00080000:1.0:1789762521.562870:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762521.562880:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762521.562924:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565781888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762521.562958:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762521.562963:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762521.562964:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762521.562969:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2456 00000100:00080000:1.0:1789762521.562973:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762521.562978:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762521.562990:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762521.563007:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565782016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762521.563018:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565781888:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762521.563030:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762521.563036:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762521.563038:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762521.563046:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2456 00000100:00080000:1.0:1789762521.563052:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762521.563057:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762521.563065:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762521.563081:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565782144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762521.563094:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762521.563096:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762521.563102:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762521.563106:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762521.563115:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2456 00000100:00080000:1.0:1789762521.563136:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789762521.563145:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762521/real 1789762521] req@ffff93f844ccad80 x1876699565781888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762521.563150:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762521.563164:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762521.563197:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565782272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762521.563198:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565781888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762521.563284:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762521.563293:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762521.563300:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565781888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762521.563330:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565782016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762521.563359:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762521.563374:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762521/real 1789762521] req@ffff93f844cc9500 x1876699565782016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762521.563389:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565782016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762521.563398:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762521.563402:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762521.563406:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565782016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762521.563413:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565782144:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762521.563436:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762521.563448:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762521/real 1789762521] req@ffff93f844ccb100 x1876699565782144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762521.563461:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565782144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762521.563484:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762521.563488:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762521.563492:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565782144:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762521.563499:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565782272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762521.563522:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762521.563534:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762521/real 1789762521] req@ffff93f844cc8e00 x1876699565782272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762521.563547:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565782272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762521.563554:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762521.563557:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762521.563559:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565782272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762521.626779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762522.650851:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762523.674805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762524.698845:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762525.722786:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762526.682748:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762526.682758:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762526.682764:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762526.682775:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2461 00000100:00080000:2.0:1789762526.682787:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762526.682795:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762526.682837:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565782400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762526.682864:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762526.682867:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762526.682868:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762526.682871:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2461 00000100:00080000:2.0:1789762526.682873:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762526.682876:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762526.682885:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762526.682895:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565782528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762526.682906:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762526.682910:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762526.682914:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789762526.682918:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565782400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762526.682919:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2461 00000100:00080000:2.0:1789762526.682921:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762526.682931:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762526.682936:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762526.682947:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565782656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762526.682954:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762526.682958:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762526.682959:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762526.682963:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2461 00000100:00080000:2.0:1789762526.682965:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762526.682968:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762526.682972:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762526.682978:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565782784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762526.682993:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762526.683034:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762526/real 1789762526] req@ffff93f8517c2d80 x1876699565782400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762526.683048:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565782400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762526.683061:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762526.683065:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762526.683067:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565782400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762526.683105:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565782528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762526.683131:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762526.683143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762526/real 1789762526] req@ffff93f8517c2a00 x1876699565782528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762526.683152:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565782528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762526.683158:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762526.683160:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762526.683163:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565782528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762526.683167:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565782656:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762526.683181:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762526.683188:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762526/real 1789762526] req@ffff93f8517c3100 x1876699565782656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762526.683195:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565782656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762526.683201:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762526.683202:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762526.683204:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565782656:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762526.683208:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565782784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762526.683219:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762526.683225:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762526/real 1789762526] req@ffff93f8517c1180 x1876699565782784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762526.683233:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565782784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762526.683238:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762526.683240:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762526.683241:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565782784:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762526.746866:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762527.770782:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762528.794857:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762529.818783:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762530.842832:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762531.802777:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762531.802820:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762531.802824:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762531.802833:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2466 00000100:00080000:2.0:1789762531.802840:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762531.802846:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762531.802879:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565782912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762531.802898:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762531.802901:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762531.802903:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762531.802905:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2466 00000100:00080000:2.0:1789762531.802908:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762531.802911:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762531.802919:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762531.802928:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565782912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762531.802929:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565783040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762531.802941:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762531.802943:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762531.802944:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762531.802947:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2466 00000100:00080000:2.0:1789762531.802949:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762531.802952:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762531.802955:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762531.802963:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565783168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762531.802971:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762531.802973:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762531.802974:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762531.802976:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2466 00000100:00080000:2.0:1789762531.802979:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762531.802981:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762531.802984:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762531.802990:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565783296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762531.802996:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762531.803062:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762531/real 1789762531] req@ffff93f8517c0380 x1876699565782912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762531.803081:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565782912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762531.803090:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762531.803094:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762531.803098:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565782912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762531.803119:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565783040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762531.803141:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762531.803152:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762531/real 1789762531] req@ffff93f8517c1500 x1876699565783040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762531.803162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565783040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762531.803169:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762531.803172:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762531.803174:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565783040:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762531.803180:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565783168:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762531.803198:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762531.803209:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762531/real 1789762531] req@ffff93f8517c2680 x1876699565783168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762531.803221:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565783168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762531.803228:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762531.803232:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762531.803234:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565783168:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762531.803239:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565783296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762531.803251:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762531.803261:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762531/real 1789762531] req@ffff93f8517c0a80 x1876699565783296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762531.803269:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565783296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762531.803279:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762531.803281:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762531.803284:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565783296:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762531.866801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762532.890780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762533.914754:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762534.938809:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762535.962760:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762536.922759:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762536.922768:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762536.922772:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762536.922781:0:8:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2471 00000100:00080000:2.0:1789762536.922791:0:8:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762536.922798:0:8:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762536.922833:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565783424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762536.922866:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762536.922870:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762536.922873:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762536.922879:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2471 00000100:00080000:2.0:1789762536.922884:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762536.922890:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762536.922902:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762536.922911:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565783424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762536.922923:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565783552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762536.922943:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762536.922948:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762536.922951:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762536.922957:0:8:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2471 00000100:00080000:2.0:1789762536.922967:0:8:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762536.922972:0:8:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762536.922979:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789762536.922992:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762536.922994:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565783680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762536.923005:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762536.923009:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762536.923011:0:8:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762536.923017:0:8:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2471 00000100:00080000:2.0:1789762536.923020:0:8:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762536.923023:0:8:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762536.923027:0:8:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762536.923043:0:8:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565783808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762536.923053:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762536/real 1789762536] req@ffff93f8517c2300 x1876699565783424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762536.923078:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565783424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762536.923088:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762536.923092:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762536.923096:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565783424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762536.923138:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565783552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762536.923165:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762536.923179:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762536/real 1789762536] req@ffff93f8517c1f80 x1876699565783552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762536.923188:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565783552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762536.923194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762536.923197:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762536.923199:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565783552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762536.923205:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565783680:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762536.923220:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762536.923228:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762536/real 1789762536] req@ffff93f8517c3b80 x1876699565783680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762536.923236:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565783680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762536.923241:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762536.923243:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762536.923245:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565783680:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762536.923249:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565783808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762536.923260:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762536.923266:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762536/real 1789762536] req@ffff93f8517c0000 x1876699565783808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762536.923274:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565783808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762536.923279:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762536.923281:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762536.923282:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565783808:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762536.986834:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762538.010793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762539.034751:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762540.058769:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762541.082792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762542.042759:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762542.042770:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762542.042774:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762542.042784:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2477 00000100:00080000:2.0:1789762542.042794:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762542.042800:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762542.042835:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565783936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762542.042856:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762542.042861:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762542.042863:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762542.042868:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2477 00000100:00080000:2.0:1789762542.042873:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762542.042878:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762542.042887:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762542.042898:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565783936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762542.042900:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565784064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762542.042911:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762542.042914:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762542.042916:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762542.042919:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2477 00000100:00080000:2.0:1789762542.042923:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762542.042926:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762542.042930:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762542.042938:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565784192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762542.042947:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762542.042949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762542.042951:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762542.042954:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2477 00000800:00000100:0.0:1789762542.042955:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762542.042958:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762542.042961:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762542.042965:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762542.042973:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565784320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762542.044078:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762542/real 1789762542] req@ffff93f8517c0000 x1876699565783936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762542.044094:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565783936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762542.044102:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762542.044105:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762542.044109:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565783936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762542.044133:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565784064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762542.044160:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762542.044173:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762542/real 1789762542] req@ffff93f8517c3b80 x1876699565784064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762542.044181:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565784064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762542.044187:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762542.044189:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762542.044191:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565784064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762542.044195:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565784192:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762542.044210:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762542.044217:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762542/real 1789762542] req@ffff93f8517c1f80 x1876699565784192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762542.044224:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565784192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762542.044234:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762542.044236:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762542.044238:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565784192:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762542.044241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565784320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762542.044251:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762542.044257:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762542/real 1789762542] req@ffff93f8517c2300 x1876699565784320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762542.044263:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565784320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762542.044268:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762542.044270:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762542.044272:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565784320:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762542.106840:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762543.130793:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762544.154815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762545.178743:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762546.202805:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762547.162719:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762547.162727:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762547.162731:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762547.162740:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2482 00000100:00080000:2.0:1789762547.162750:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762547.162758:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762547.162795:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565784448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762547.162822:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762547.162825:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762547.162827:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762547.162830:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2482 00000100:00080000:2.0:1789762547.162834:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762547.162838:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762547.162846:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762547.162851:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565784448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762547.162861:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565784576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762547.162870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762547.162873:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762547.162874:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762547.162877:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2482 00000100:00080000:2.0:1789762547.162881:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762547.162885:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762547.162890:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762547.162903:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565784704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762547.162910:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762547.162922:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762547.162926:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762547.162929:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762547.162934:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2482 00000100:00080000:2.0:1789762547.162938:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762547.162941:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762547.162945:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762547.162957:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565784832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762547.162963:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762547/real 1789762547] req@ffff93f8517c0a80 x1876699565784448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762547.162980:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565784448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762547.162988:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762547.162991:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762547.162995:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565784448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762547.163013:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565784576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762547.163029:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762547.163038:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762547/real 1789762547] req@ffff93f8517c2680 x1876699565784576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762547.163046:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565784576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762547.163052:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762547.163054:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762547.163056:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565784576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762547.163060:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565784704:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762547.163070:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762547.163076:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762547/real 1789762547] req@ffff93f8517c1500 x1876699565784704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762547.163084:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565784704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762547.163089:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762547.163091:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762547.163092:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565784704:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762547.163096:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565784832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762547.163106:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762547.163112:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762547/real 1789762547] req@ffff93f8517c0380 x1876699565784832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762547.163120:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565784832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762547.163125:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762547.163127:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762547.163129:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565784832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762547.226772:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762548.250837:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762549.274823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762550.298780:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762551.322820:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762552.282743:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762552.282756:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762552.282761:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762552.282771:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2487 00000100:00080000:2.0:1789762552.282781:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762552.282789:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762552.282825:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565784960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762552.282861:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762552.282865:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762552.282868:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762552.282873:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2487 00000100:00080000:2.0:1789762552.282877:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789762552.282891:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565784960:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762552.282891:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762552.282905:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762552.282934:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565785088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762552.282947:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762552.282951:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762552.282953:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762552.282958:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2487 00000800:00000100:0.0:1789762552.282964:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762552.282964:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762552.282973:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762552.282989:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762552.283003:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565785216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762552.283014:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762552.283018:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762552.283020:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789762552.283023:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762552/real 1789762552] req@ffff93f8517c1180 x1876699565784960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762552.283028:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2487 00000100:00080000:2.0:1789762552.283035:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762552.283044:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762552.283050:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762552.283052:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565784960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762552.283064:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565785344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762552.283079:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762552.283086:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762552.283091:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565784960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762552.283114:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565785088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762552.283132:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762552.283142:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762552/real 1789762552] req@ffff93f8517c3100 x1876699565785088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762552.283151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565785088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762552.283161:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762552.283163:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762552.283165:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565785088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762552.283169:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565785216:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762552.283180:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762552.283186:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762552/real 1789762552] req@ffff93f8517c2a00 x1876699565785216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762552.283203:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565785216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762552.283208:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762552.283210:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762552.283212:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565785216:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762552.283216:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565785344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762552.283227:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762552.283234:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762552/real 1789762552] req@ffff93f8517c2d80 x1876699565785344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762552.283258:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565785344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762552.283266:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762552.283277:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762552.283281:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565785344:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762552.346787:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762553.370830:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762554.394788:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762555.418797:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762556.442784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762557.402757:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762557.402768:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762557.402773:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762557.402784:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2492 00000100:00080000:2.0:1789762557.402795:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762557.402803:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762557.402841:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565785472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762557.402867:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762557.402871:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762557.402873:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762557.402877:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2492 00000100:00080000:2.0:1789762557.402881:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762557.402886:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762557.402895:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762557.402904:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565785472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762557.402912:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565785600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762557.402925:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762557.402928:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762557.402931:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762557.402935:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2492 00000100:00080000:2.0:1789762557.402940:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762557.402944:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762557.402950:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789762557.402960:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762557.402965:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565785728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762557.402977:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762557.402982:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762557.402983:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762557.402987:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2492 00000100:00080000:2.0:1789762557.402992:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762557.402995:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762557.403001:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762557.403015:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762557/real 1789762557] req@ffff93f8517c0e00 x1876699565785472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762557.403015:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565785856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762557.403054:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565785472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762557.403063:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762557.403066:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762557.403070:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565785472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762557.403087:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565785600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762557.403103:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762557.403111:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762557/real 1789762557] req@ffff93f8517c3480 x1876699565785600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762557.403120:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565785600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762557.403129:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762557.403133:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762557.403136:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565785600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762557.403142:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565785728:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762557.403163:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762557.403175:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762557/real 1789762557] req@ffff93f8517c3800 x1876699565785728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762557.403186:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565785728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762557.403192:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762557.403193:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762557.403195:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565785728:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762557.403214:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565785856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762557.403230:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762557.403238:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762557/real 1789762557] req@ffff93f8517c1880 x1876699565785856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762557.403246:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565785856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762557.403251:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762557.403253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762557.403255:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565785856:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762557.466800:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762558.490818:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762559.514771:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762560.538803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762561.562776:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762562.522848:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762562.522858:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762562.522863:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762562.522875:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2497 00000100:00080000:3.0:1789762562.522885:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762562.522961:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762562.523011:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565785984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762562.523045:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762562.523050:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762562.523051:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762562.523056:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2497 00000100:00080000:3.0:1789762562.523060:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762562.523076:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762562.523085:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762562.523099:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565786112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762562.523108:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762562.523111:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762562.523113:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762562.523117:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2497 00000100:00080000:3.0:1789762562.523120:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762562.523123:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762562.523140:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762562.523150:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565786240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762562.523173:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762562.523177:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762562.523179:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762562.523183:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2497 00000100:00080000:3.0:1789762562.523186:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762562.523190:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762562.523195:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762562.523208:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565786368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762562.523305:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565785984:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1789762562.523365:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762562.523413:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762562/real 1789762562] req@ffff93f844d67480 x1876699565785984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762562.523430:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565785984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762562.523441:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762562.523445:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762562.523448:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565785984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762562.523455:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565786112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762562.523476:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762562.523498:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762562/real 1789762562] req@ffff93f844d67800 x1876699565786112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762562.523512:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565786112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762562.523521:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762562.523524:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762562.523527:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565786112:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762562.523534:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565786240:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762562.523554:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762562.523566:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762562/real 1789762562] req@ffff93f844d67100 x1876699565786240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762562.523578:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565786240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762562.523586:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762562.523589:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762562.523592:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565786240:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762562.523599:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565786368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762562.523619:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762562.523631:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762562/real 1789762562] req@ffff93f844d66a00 x1876699565786368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762562.523723:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565786368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762562.523730:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762562.523733:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762562.523736:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565786368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762562.586754:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762563.610817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762564.634814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762565.658873:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762566.682810:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762567.642764:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762567.642775:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762567.642781:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762567.642796:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2502 00000100:00080000:3.0:1789762567.642809:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762567.642817:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762567.642858:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565786496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762567.642899:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762567.642904:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762567.642906:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762567.642912:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2502 00000100:00080000:3.0:1789762567.642917:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762567.642922:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762567.642932:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565786496:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789762567.642934:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762567.642958:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565786624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762567.642972:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762567.642976:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762567.642977:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762567.642981:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2502 00000100:00080000:3.0:1789762567.642999:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762567.643003:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762567.643008:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789762567.643014:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762567.643063:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762567/real 1789762567] req@ffff93f844d65c00 x1876699565786496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762567.643096:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565786496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762567.643108:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762567.643111:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762567.643115:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565786496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762567.643158:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565786624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762567.643179:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762567.643195:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762567/real 1789762567] req@ffff93f844d64e00 x1876699565786624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762567.643205:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565786624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762567.643214:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762567.643216:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762567.643219:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565786624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762567.643246:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565786752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762567.643295:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565786752:192.168.203.147@tcp:8:kworker.0 00000100:00080000:3.0:1789762567.643321:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762567.643327:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:2.0:1789762567.643328:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762567.643335:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762567.643345:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2502 00000100:00100000:2.0:1789762567.643349:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762567/real 1789762567] req@ffff93f844d64000 x1876699565786752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762567.643357:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762567.643367:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762567.643370:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565786752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1789762567.643384:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00080000:2.0:1789762567.643387:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762567.643397:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762567.643400:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565786752:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762567.643413:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565786880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762567.643448:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565786880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762567.643475:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762567.643490:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762567/real 1789762567] req@ffff93f844d65500 x1876699565786880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762567.643503:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565786880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762567.643513:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762567.643515:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762567.643519:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565786880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762567.706798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762568.730831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762569.754895:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762570.778801:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762571.802789:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762572.762741:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762572.762750:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762572.762754:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762572.762762:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2507 00000100:00080000:3.0:1789762572.762771:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762572.762777:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762572.762809:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565787008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762572.762829:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762572.762832:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762572.762833:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762572.762836:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2507 00000100:00080000:3.0:1789762572.762838:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762572.762841:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762572.762848:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762572.762859:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565787136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762572.762868:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789762572.762869:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565787008:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762572.762871:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762572.762874:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762572.762882:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2507 00000100:00080000:3.0:1789762572.762884:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762572.762886:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762572.762890:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762572.762900:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565787264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762572.762906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762572.762909:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762572.762910:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762572.762913:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2507 00000100:00080000:3.0:1789762572.762924:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762572.762927:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762572.762930:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762572.762939:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565787392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762572.762941:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762572.763002:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762572/real 1789762572] req@ffff93f844d64380 x1876699565787008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762572.763023:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565787008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762572.763035:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762572.763039:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762572.763043:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565787008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762572.763068:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565787136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762572.763106:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762572.763119:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762572/real 1789762572] req@ffff93f844d66d80 x1876699565787136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762572.763131:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565787136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762572.763139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762572.763143:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762572.763146:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565787136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762572.763153:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565787264:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762572.763172:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762572.763182:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762572/real 1789762572] req@ffff93f844d64a80 x1876699565787264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762572.763194:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565787264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762572.763202:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762572.763205:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762572.763208:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565787264:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762572.763214:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565787392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762572.763230:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762572.763240:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762572/real 1789762572] req@ffff93f844d65f80 x1876699565787392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762572.763252:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565787392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762572.763259:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762572.763263:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762572.763266:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565787392:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762572.826797:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762573.850830:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762574.874871:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762575.898812:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762576.922830:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762577.882784:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762577.884071:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762577.884075:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762577.884085:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2512 00000100:00080000:1.0:1789762577.884095:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762577.884102:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762577.884139:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565787520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762577.884166:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762577.884170:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762577.884172:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762577.884177:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2512 00000100:00080000:1.0:1789762577.884180:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762577.884185:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762577.884194:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762577.884212:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565787648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762577.884212:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565787520:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762577.884230:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762577.884233:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762577.884235:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762577.884239:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2512 00000100:00080000:1.0:1789762577.884242:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762577.884247:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762577.884253:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762577.884266:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565787776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762577.884276:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1789762577.884280:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762577.884282:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762577.884286:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762577.884291:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2512 00000100:00080000:1.0:1789762577.884296:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762577.884301:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762577.884307:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762577.884320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565787904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762577.884334:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762577/real 1789762577] req@ffff93f844cca680 x1876699565787520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762577.884365:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565787520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762577.884376:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762577.884380:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762577.884384:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565787520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762577.884406:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565787648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762577.884429:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762577.884441:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762577/real 1789762577] req@ffff93f844cc9c00 x1876699565787648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762577.884454:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565787648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762577.884462:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762577.884466:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762577.884469:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565787648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762577.884475:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565787776:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762577.884495:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762577.884505:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762577/real 1789762577] req@ffff93f844ccaa00 x1876699565787776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762577.884516:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565787776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762577.884524:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762577.884527:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762577.884531:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565787776:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762577.884536:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565787904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762577.884556:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762577.884567:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762577/real 1789762577] req@ffff93f844ccb480 x1876699565787904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762577.884579:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565787904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762577.884587:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762577.884590:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762577.884594:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565787904:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762577.946822:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762578.970789:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762579.994835:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762581.018790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762582.042776:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762583.002722:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762583.002730:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762583.002733:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762583.002741:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2517 00000100:00080000:2.0:1789762583.002750:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762583.002756:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762583.002787:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565788032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762583.002813:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762583.002818:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762583.002820:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762583.002825:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2517 00000100:00080000:2.0:1789762583.002829:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762583.002835:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762583.002844:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762583.002854:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565788032:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762583.002858:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565788160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762583.002868:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762583.002870:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762583.002871:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762583.002874:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2517 00000100:00080000:2.0:1789762583.002877:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762583.002880:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762583.002884:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762583.002892:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565788288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762583.002900:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762583.002902:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762583.002904:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762583.002908:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2517 00000100:00080000:2.0:1789762583.002912:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762583.002917:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762583.002922:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762583.002935:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565788416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762583.002946:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565788160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762583.002970:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565788288:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762583.002988:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565788416:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762583.066851:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762584.090798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762585.114831:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789762586.074906:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789762586.074943:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789762586.074956:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762586.075021:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762586.075031:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762586.075039:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789762586.075190:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762583/real 1789762586] req@ffff93f8517c0e00 x1876699565788416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762586.075239:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565788416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762586.075258:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762586.075265:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762586.075271:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565788416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762586.075284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762583/real 1789762586] req@ffff93f8517c3800 x1876699565788160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762586.075301:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565788160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762586.075312:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762586.075317:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762586.075322:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565788160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762586.075332:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762583/real 1789762586] req@ffff93f8517c3480 x1876699565788288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762586.075350:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565788288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762586.075361:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762586.075366:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762586.075391:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565788288:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762586.075403:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762583/real 1789762586] req@ffff93f8517c1880 x1876699565788032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762586.075421:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565788032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762586.075432:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762586.075436:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762586.075440:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565788032:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789762586.138798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762587.162823:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762588.122768:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762588.122780:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762588.122785:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762588.122797:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2523 00000100:00080000:0.0:1789762588.122811:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762588.122822:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762588.122864:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565788544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762588.122911:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762588.122915:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762588.122918:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762588.122925:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2523 00000100:00080000:0.0:1789762588.122929:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762588.122935:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762588.122946:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762588.122959:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565788544:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762588.122965:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565788672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762588.122995:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762588.122999:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762588.123001:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762588.123007:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2523 00000100:00080000:0.0:1789762588.123011:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762588.123015:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762588.123022:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762588.123039:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565788800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762588.123050:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762588.123053:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762588.123056:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762588.123060:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2523 00000100:00080000:0.0:1789762588.123064:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762588.123069:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1789762588.123076:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1789762588.123083:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762588.123104:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565788928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762588.123143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762588/real 1789762588] req@ffff93f850d43800 x1876699565788544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762588.123170:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565788544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762588.123186:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762588.123191:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762588.123196:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565788544:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762588.123243:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565788672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762588.123272:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762588.123288:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762588/real 1789762588] req@ffff93f850d43480 x1876699565788672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762588.123301:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565788672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762588.123311:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762588.123315:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762588.123319:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565788672:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762588.123326:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565788800:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762588.123350:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762588.123362:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762588/real 1789762588] req@ffff93f850d41500 x1876699565788800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762588.123373:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565788800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762588.123382:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762588.123386:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762588.123390:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565788800:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762588.123397:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565788928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762588.123428:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762588.123442:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762588/real 1789762588] req@ffff93f850d41f80 x1876699565788928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762588.123455:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565788928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762588.123464:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762588.123467:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762588.123472:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565788928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762588.186784:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762589.210797:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762590.234824:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762591.258796:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762592.282959:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762593.242752:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762593.242764:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762593.242770:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762593.242783:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2528 00000100:00080000:0.0:1789762593.242796:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762593.242803:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762593.242858:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565789056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762593.242890:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762593.242895:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762593.242897:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762593.242902:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2528 00000100:00080000:0.0:1789762593.242908:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762593.242912:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762593.242923:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762593.242928:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565789056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762593.242959:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565789184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762593.242973:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762593.242977:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762593.242980:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762593.242985:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2528 00000100:00080000:0.0:1789762593.243010:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762593.243013:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762593.243019:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:1.0:1789762593.243020:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762593.243035:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565789312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762593.243044:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762593.243047:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762593.243049:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762593.243052:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2528 00000100:00080000:0.0:1789762593.243055:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762593.243058:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762593.243063:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762593.243070:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762593/real 1789762593] req@ffff93f850d43100 x1876699565789056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762593.243073:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565789440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762593.243100:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565789056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762593.243111:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762593.243116:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762593.243119:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565789056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762593.243141:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565789184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762593.243158:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762593.243167:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762593/real 1789762593] req@ffff93f850d41180 x1876699565789184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762593.243175:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565789184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762593.243181:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762593.243183:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762593.243185:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565789184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762593.243189:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565789312:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762593.243201:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762593.243208:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762593/real 1789762593] req@ffff93f850d40000 x1876699565789312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762593.243216:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565789312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762593.243221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762593.243223:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762593.243225:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565789312:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762593.243228:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565789440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762593.243237:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762593.243243:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762593/real 1789762593] req@ffff93f850d42680 x1876699565789440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762593.243251:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565789440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762593.243256:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762593.243258:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762593.243259:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565789440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762593.306809:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762594.330816:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762595.354801:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762596.378793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762597.402785:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762598.362740:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762598.362747:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762598.362751:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762598.362759:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2533 00000100:00080000:1.0:1789762598.362767:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762598.362773:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762598.362804:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565789568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762598.362828:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762598.362832:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762598.362834:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762598.362838:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2533 00000100:00080000:1.0:1789762598.362842:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762598.362846:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762598.362856:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762598.362866:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565789568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762598.362868:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565789696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762598.362878:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762598.362881:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762598.362883:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762598.362886:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2533 00000100:00080000:1.0:1789762598.362889:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762598.362905:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762598.362911:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762598.362920:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565789824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762598.362930:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762598.362933:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762598.362935:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789762598.362937:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762598.362939:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2533 00000100:00080000:1.0:1789762598.362942:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762598.362946:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762598.362950:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762598.362959:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565789952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762598.362975:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762598/real 1789762598] req@ffff93f844ccb800 x1876699565789568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762598.362992:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565789568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762598.363003:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762598.363006:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762598.363009:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565789568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762598.363027:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565789696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762598.363045:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762598.363055:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762598/real 1789762598] req@ffff93f844cc8700 x1876699565789696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762598.363063:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565789696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762598.363068:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762598.363070:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762598.363073:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565789696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762598.363076:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565789824:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762598.363087:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762598.363093:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762598/real 1789762598] req@ffff93f844ccbb80 x1876699565789824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762598.363101:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565789824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762598.363106:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762598.363107:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762598.363109:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565789824:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762598.363112:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565789952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762598.363122:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762598.363128:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762598/real 1789762598] req@ffff93f844cc8000 x1876699565789952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762598.363135:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565789952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762598.363140:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762598.363142:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762598.363144:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565789952:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762598.426754:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762599.450824:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762600.474779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762601.498793:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762602.522808:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762603.482743:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762603.482752:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762603.482758:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762603.482771:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2538 00000100:00080000:1.0:1789762603.482784:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762603.482792:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762603.482834:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565790080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762603.482871:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762603.482876:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762603.482879:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762603.482886:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2538 00000100:00080000:1.0:1789762603.482892:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762603.482901:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762603.482911:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565790080:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762603.482917:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762603.482947:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565790208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762603.482962:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762603.482966:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762603.482970:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762603.482974:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2538 00000100:00080000:1.0:1789762603.482980:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762603.482984:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762603.482989:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762603.483006:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565790336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789762603.483006:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762603.483020:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762603.483023:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762603.483025:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762603.483031:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2538 00000100:00080000:1.0:1789762603.483035:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762603.483039:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762603.483043:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762603.483052:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565790464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762603.483132:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762603/real 1789762603] req@ffff93f844cc8380 x1876699565790080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762603.483314:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565790080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762603.483330:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762603.483346:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762603.483353:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565790080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762603.483381:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565790208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762603.483408:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762603.483422:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762603/real 1789762603] req@ffff93f844cca300 x1876699565790208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762603.483435:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565790208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762603.483445:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762603.483449:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762603.483452:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565790208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762603.483459:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565790336:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762603.483480:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762603.483492:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762603/real 1789762603] req@ffff93f844cc9f80 x1876699565790336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762603.483504:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565790336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762603.483519:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762603.483523:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762603.483527:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565790336:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762603.483534:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565790464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762603.483560:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762603.483571:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762603/real 1789762603] req@ffff93f844cc9880 x1876699565790464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762603.483583:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565790464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762603.483592:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762603.483596:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762603.483600:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565790464:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762603.546778:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762604.570841:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762605.594783:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762606.618820:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762607.642775:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762608.602737:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762608.602748:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762608.602753:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762608.602764:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2543 00000100:00080000:1.0:1789762608.602775:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762608.602783:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762608.602837:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565790592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762608.602860:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762608.602864:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762608.602865:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762608.602870:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2543 00000100:00080000:1.0:1789762608.602874:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762608.602878:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762608.602888:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762608.602903:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565790720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762608.602911:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565790592:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762608.602913:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762608.602917:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762608.602919:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762608.602922:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2543 00000100:00080000:1.0:1789762608.602926:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762608.602929:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762608.602934:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762608.602945:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565790848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762608.602955:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762608.602958:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762608.602960:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762608.602963:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2543 00000100:00080000:1.0:1789762608.602968:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762608.602972:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762608.602976:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789762608.602978:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762608.602985:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565790976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762608.603023:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762608/real 1789762608] req@ffff93f844cc8a80 x1876699565790592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762608.603042:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565790592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762608.603054:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762608.603059:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762608.603063:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565790592:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762608.603089:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565790720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762608.603116:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762608.603131:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762608/real 1789762608] req@ffff93f844cc9180 x1876699565790720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762608.603145:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565790720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762608.603151:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762608.603153:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762608.603155:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565790720:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762608.603160:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565790848:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762608.603177:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762608.603184:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762608/real 1789762608] req@ffff93f844cc9880 x1876699565790848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762608.603192:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565790848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762608.603197:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762608.603199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762608.603201:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565790848:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762608.603205:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565790976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762608.603216:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762608.603222:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762608/real 1789762608] req@ffff93f844cc9f80 x1876699565790976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762608.603230:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565790976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762608.603235:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762608.603236:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762608.603238:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565790976:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762608.666801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762609.690795:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762610.714775:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762611.738767:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762612.762835:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762613.722755:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762613.722763:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762613.722767:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762613.722775:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2548 00000100:00080000:2.0:1789762613.722804:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762613.722812:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762613.722849:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565791104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762613.722871:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762613.722874:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762613.722876:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762613.722879:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2548 00000100:00080000:2.0:1789762613.722882:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762613.722885:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762613.722893:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762613.722904:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565791104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762613.722905:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565791232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762613.722919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762613.722921:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762613.722922:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762613.722925:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2548 00000100:00080000:2.0:1789762613.722928:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762613.722930:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762613.722933:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762613.722941:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565791360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762613.722950:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762613.722952:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762613.722954:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762613.722956:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2548 00000100:00080000:2.0:1789762613.722959:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762613.722962:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762613.722964:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789762613.722970:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762613.722973:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565791488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762613.723019:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762613/real 1789762613] req@ffff93f8517c2d80 x1876699565791104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762613.723041:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565791104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762613.723052:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762613.723057:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762613.723060:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565791104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762613.723081:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565791232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762613.723099:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762613.723109:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762613/real 1789762613] req@ffff93f8517c2a00 x1876699565791232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762613.723117:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565791232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762613.723123:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762613.723133:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762613.723136:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565791232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762613.723140:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565791360:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762613.723156:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762613.723167:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762613/real 1789762613] req@ffff93f8517c3100 x1876699565791360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762613.723180:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565791360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762613.723188:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762613.723192:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762613.723195:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565791360:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762613.723202:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565791488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762613.723223:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762613.723234:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762613/real 1789762613] req@ffff93f8517c1180 x1876699565791488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762613.723243:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565791488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762613.723249:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762613.723251:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762613.723253:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565791488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762613.786790:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762614.810871:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762615.834780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762616.858778:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762617.882772:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762618.842718:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762618.842726:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762618.842730:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762618.842738:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2553 00000100:00080000:2.0:1789762618.842758:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762618.842765:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762618.842797:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565791616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762618.842821:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762618.842825:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762618.842827:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762618.842833:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2553 00000100:00080000:2.0:1789762618.842837:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762618.842842:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762618.842851:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565791616:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789762618.842853:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762618.842871:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565791744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762618.842883:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762618.842886:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762618.842888:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762618.842891:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2553 00000100:00080000:2.0:1789762618.842895:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762618.842898:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762618.842902:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789762618.842907:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762618.842913:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565791872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762618.842921:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762618.842924:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762618.842925:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762618.842927:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2553 00000100:00080000:2.0:1789762618.842929:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762618.842932:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762618.842935:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762618.842941:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565792000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762618.842957:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762618/real 1789762618] req@ffff93f8517c0380 x1876699565791616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762618.842974:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565791616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762618.842983:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762618.842985:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762618.842988:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565791616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762618.843006:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565791744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762618.843023:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762618.843031:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762618/real 1789762618] req@ffff93f8517c1500 x1876699565791744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762618.843040:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565791744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762618.843045:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762618.843047:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762618.843049:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565791744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762618.843053:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565791872:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762618.843063:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762618.843069:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762618/real 1789762618] req@ffff93f8517c2680 x1876699565791872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762618.843076:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565791872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762618.843081:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762618.843083:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762618.843085:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565791872:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762618.843088:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565792000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762618.843098:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762618.843104:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762618/real 1789762618] req@ffff93f8517c0a80 x1876699565792000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762618.843111:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565792000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762618.843120:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762618.843122:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762618.843124:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565792000:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762618.906806:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762619.930798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762620.954863:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762621.978856:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762623.002791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762623.962755:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762623.962765:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762623.962771:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762623.962781:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2558 00000100:00080000:0.0:1789762623.962791:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762623.962799:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762623.962840:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565792128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762623.962863:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762623.962866:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762623.962868:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762623.962871:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2558 00000100:00080000:0.0:1789762623.962875:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762623.962879:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762623.962886:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762623.962893:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565792128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762623.962897:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565792256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762623.962906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762623.962908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762623.962909:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762623.962913:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2558 00000100:00080000:0.0:1789762623.962917:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762623.962921:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762623.962926:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762623.962939:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565792384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762623.962951:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1789762623.962951:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762623.962958:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762623.962962:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762623.962968:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2558 00000100:00080000:0.0:1789762623.962973:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762623.962978:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762623.962984:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762623.962994:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762623/real 1789762623] req@ffff93f850d41c00 x1876699565792128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762623.963000:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565792512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762623.963015:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565792128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762623.963025:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762623.963028:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762623.963033:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565792128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762623.963051:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565792256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762623.963071:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762623.963085:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762623/real 1789762623] req@ffff93f850d40e00 x1876699565792256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762623.963099:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565792256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762623.963108:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762623.963111:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762623.963115:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565792256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762623.963121:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565792384:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762623.963143:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762623.963155:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762623/real 1789762623] req@ffff93f850d42300 x1876699565792384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762623.963169:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565792384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762623.963178:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762623.963181:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762623.963185:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565792384:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762623.963190:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565792512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762623.963210:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762623.963221:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762623/real 1789762623] req@ffff93f850d40380 x1876699565792512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762623.963229:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565792512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762623.963235:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762623.963236:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762623.963239:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565792512:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762624.026826:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762625.050802:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762626.074838:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762627.098813:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762628.122854:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762629.082858:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762629.082869:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762629.082873:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762629.082884:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2564 00000100:00080000:0.0:1789762629.082894:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762629.082901:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762629.082947:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565792640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762629.082977:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762629.082981:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762629.082984:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762629.082989:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2564 00000100:00080000:0.0:1789762629.082993:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762629.082998:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762629.083007:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762629.083019:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565792640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762629.083025:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565792768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762629.083038:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762629.083043:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762629.083056:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762629.083061:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2564 00000100:00080000:0.0:1789762629.083066:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762629.083071:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762629.083077:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762629.083092:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565792896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789762629.083103:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762629.083105:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762629.083111:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762629.083113:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762629.083119:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2564 00000100:00080000:0.0:1789762629.083124:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762629.083129:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762629.083135:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762629.083148:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565793024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762629.083158:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762629/real 1789762629] req@ffff93f850d43b80 x1876699565792640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762629.083182:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565792640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762629.083194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762629.083199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762629.083204:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565792640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762629.083227:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565792768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762629.083263:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762629.083277:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762629/real 1789762629] req@ffff93f850d40700 x1876699565792768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762629.083300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565792768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762629.083317:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762629.083321:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762629.083326:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565792768:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762629.083334:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565792896:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762629.083355:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762629.083368:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762629/real 1789762629] req@ffff93f850d40380 x1876699565792896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762629.083382:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565792896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762629.083398:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762629.083406:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762629.083411:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565792896:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762629.083418:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565793024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762629.083439:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762629.083451:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762629/real 1789762629] req@ffff93f850d42300 x1876699565793024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762629.083459:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565793024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762629.083465:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762629.083467:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762629.083469:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565793024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762629.146896:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762630.170903:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762631.194811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762632.218782:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762633.242766:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762634.202734:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762634.202742:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762634.202746:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762634.202754:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2569 00000100:00080000:1.0:1789762634.202762:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762634.202768:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762634.202801:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565793152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762634.202826:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762634.202829:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762634.202831:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762634.202834:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2569 00000100:00080000:1.0:1789762634.202838:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762634.202841:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762634.202848:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762634.202861:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565793152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762634.202861:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565793280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762634.202876:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762634.202879:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762634.202880:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762634.202883:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2569 00000100:00080000:1.0:1789762634.202885:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762634.202887:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762634.202890:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762634.202899:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565793408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762634.202905:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762634.202908:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762634.202909:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762634.202912:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2569 00000100:00080000:1.0:1789762634.202921:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789762634.202925:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762634.202929:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762634.202933:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762634.202942:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565793536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762634.202966:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762634/real 1789762634] req@ffff93f844cca300 x1876699565793152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762634.202985:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565793152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762634.203003:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762634.203007:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762634.203012:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565793152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762634.203035:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565793280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762634.203054:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762634.203064:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762634/real 1789762634] req@ffff93f844cc8380 x1876699565793280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762634.203072:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565793280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762634.203078:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762634.203080:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762634.203082:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565793280:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762634.203086:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565793408:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762634.203096:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762634.203102:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762634/real 1789762634] req@ffff93f844cc8000 x1876699565793408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762634.203110:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565793408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762634.203115:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762634.203117:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762634.203119:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565793408:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762634.203122:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565793536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762634.203134:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762634.203140:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762634/real 1789762634] req@ffff93f844ccbb80 x1876699565793536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762634.203147:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565793536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762634.203152:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762634.203154:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762634.203155:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565793536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762634.266803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762635.290788:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762636.314780:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762637.338772:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762638.362778:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762639.322755:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762639.322766:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762639.322771:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762639.322782:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2574 00000100:00080000:3.0:1789762639.322792:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762639.322800:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762639.322836:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565793664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762639.322865:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762639.322870:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762639.322872:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762639.322877:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2574 00000100:00080000:3.0:1789762639.322882:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762639.322887:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762639.322893:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565793664:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789762639.322899:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762639.322916:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565793792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762639.322939:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762639.322943:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762639.322945:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762639.322949:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2574 00000100:00080000:3.0:1789762639.322954:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762639.322958:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1789762639.322963:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1789762639.322966:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762639.322982:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565793920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762639.322991:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762639.322994:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762639.322995:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789762639.322999:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762639/real 1789762639] req@ffff93f844d65180 x1876699565793664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762639.323001:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2574 00000100:00080000:3.0:1789762639.323007:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762639.323013:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762639.323023:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565793664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762639.323045:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762639.323059:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762639.323061:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565793664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762639.323083:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565793792:192.168.203.147@tcp:38:kworker.0 00000008:00100000:3.0:1789762639.323084:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762639.323101:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565794048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789762639.323118:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762639.323138:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762639/real 1789762639] req@ffff93f844d66300 x1876699565793792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762639.323153:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565793792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762639.323160:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762639.323163:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762639.323166:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565793792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762639.323171:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565793920:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762639.323188:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762639.323198:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762639/real 1789762639] req@ffff93f844d65f80 x1876699565793920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762639.323216:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565793920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762639.323225:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762639.323229:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762639.323233:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565793920:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762639.323254:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565794048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762639.323279:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762639.323289:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762639/real 1789762639] req@ffff93f844d64a80 x1876699565794048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762639.323300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565794048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762639.323307:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762639.323310:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762639.323321:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565794048:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762639.386750:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762640.410799:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762641.434750:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762642.458766:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762643.482806:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762644.442765:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762644.442776:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762644.442781:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762644.442791:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2579 00000100:00080000:1.0:1789762644.442800:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762644.442806:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762644.442846:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565794176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762644.442875:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762644.442880:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762644.442882:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762644.442886:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2579 00000100:00080000:1.0:1789762644.442891:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762644.442896:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762644.442905:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762644.442918:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565794304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762644.442925:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762644.442928:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762644.442930:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762644.442933:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2579 00000100:00080000:1.0:1789762644.442936:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762644.442939:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762644.442944:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762644.442955:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565794432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762644.442960:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565794176:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762644.442964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762644.442969:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762644.442970:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762644.442972:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2579 00000100:00080000:1.0:1789762644.442974:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762644.442976:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762644.442979:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762644.442998:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565794560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762644.443033:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762644.443069:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762644/real 1789762644] req@ffff93f844cc8700 x1876699565794176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762644.443082:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565794176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762644.443090:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762644.443093:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762644.443096:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565794176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762644.443100:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565794304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762644.443114:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762644.443122:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762644/real 1789762644] req@ffff93f844ccb800 x1876699565794304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762644.443130:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565794304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762644.443136:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762644.443138:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762644.443140:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565794304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762644.443157:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565794432:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762644.443168:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762644.443174:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762644/real 1789762644] req@ffff93f844ccb480 x1876699565794432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762644.443181:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565794432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762644.443197:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762644.443199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762644.443201:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565794432:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762644.443204:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565794560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762644.443214:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762644.443219:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762644/real 1789762644] req@ffff93f844ccaa00 x1876699565794560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762644.443226:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565794560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762644.443236:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762644.443238:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762644.443240:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565794560:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762644.506782:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762645.530810:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762646.554766:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762647.578785:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762648.602785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762649.562745:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762649.562759:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762649.562765:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762649.562780:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2584 00000100:00080000:3.0:1789762649.562791:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762649.562798:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762649.562865:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565794688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762649.562902:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762649.562908:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762649.562911:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762649.562919:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2584 00000100:00080000:3.0:1789762649.562925:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762649.562933:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762649.562948:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762649.562952:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565794688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762649.562975:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565794816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762649.562989:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762649.562993:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762649.562995:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762649.563003:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2584 00000100:00080000:3.0:1789762649.563007:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762649.563012:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762649.563021:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762649.563032:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565794944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762649.563058:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762649.563062:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762649.563064:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762649.563069:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2584 00000100:00080000:3.0:1789762649.563074:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762649.563079:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762649.563083:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762649.563096:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565794816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762649.563097:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565795072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762649.563161:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565794944:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762649.563211:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565795072:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762649.626815:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762650.650769:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762651.674773:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789762652.634991:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789762652.635043:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789762652.635062:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762652.635122:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762652.635134:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762652.635146:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789762652.635203:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762649/real 1789762652] req@ffff93f844d64000 x1876699565795072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762652.635237:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565795072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762652.635260:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762652.635267:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762652.635273:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565795072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762652.635282:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762649/real 1789762652] req@ffff93f844d64380 x1876699565794816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762652.635295:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565794816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762652.635304:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762652.635309:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762652.635312:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565794816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762652.635336:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762649/real 1789762652] req@ffff93f844d65500 x1876699565794944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762652.635350:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565794944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762652.635359:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762652.635363:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762652.635367:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565794944:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762652.635374:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762649/real 1789762652] req@ffff93f844d66d80 x1876699565794688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762652.635387:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565794688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762652.635395:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762652.635398:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762652.635402:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565794688:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789762652.698767:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762653.722757:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762654.682788:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762654.682799:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762654.682804:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762654.682820:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2589 00000100:00080000:3.0:1789762654.682829:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762654.682838:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762654.682873:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565795200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762654.682911:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762654.682916:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762654.682918:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762654.682922:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2589 00000100:00080000:3.0:1789762654.682926:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762654.682931:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762654.682940:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762654.682954:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565795200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762654.682959:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565795328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762654.682972:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762654.682976:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762654.682978:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762654.682983:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2589 00000100:00080000:3.0:1789762654.682987:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762654.682991:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762654.682997:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762654.683010:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565795456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762654.683020:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762654.683028:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762654.683033:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762654.683036:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762654.683041:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2589 00000100:00080000:3.0:1789762654.683046:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762654.683051:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762654.683057:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762654.683066:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762654/real 1789762654] req@ffff93f844d64e00 x1876699565795200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762654.683074:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565795584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762654.683096:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565795200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762654.683110:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762654.683121:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762654.683126:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565795200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762654.683156:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565795328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762654.683177:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762654.683192:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762654/real 1789762654] req@ffff93f844d65c00 x1876699565795328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762654.683204:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565795328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762654.683213:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762654.683217:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762654.683221:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565795328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762654.683227:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565795456:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762654.683255:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762654.683268:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762654/real 1789762654] req@ffff93f844d66a00 x1876699565795456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762654.683280:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565795456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762654.683288:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762654.683291:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762654.683294:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565795456:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762654.683306:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565795584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762654.683326:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762654.683338:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762654/real 1789762654] req@ffff93f844d67100 x1876699565795584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762654.683350:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565795584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762654.683358:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762654.683361:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762654.683364:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565795584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762654.746778:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762655.770786:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762656.794785:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762657.818791:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762658.842786:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762659.802752:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762659.802760:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762659.802765:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762659.802773:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2594 00000100:00080000:3.0:1789762659.802780:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762659.802787:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762659.802822:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565795712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762659.802865:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762659.802870:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762659.802872:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762659.802877:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2594 00000100:00080000:3.0:1789762659.802881:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762659.802887:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762659.802897:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762659.802924:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565795712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762659.802925:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565795840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762659.802941:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762659.802943:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762659.802945:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762659.802948:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2594 00000100:00080000:3.0:1789762659.802951:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762659.802954:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762659.802959:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762659.802969:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565795968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762659.802976:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762659.802979:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:1.0:1789762659.802982:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762659.802982:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762659.802989:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2594 00000100:00080000:3.0:1789762659.802993:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762659.802996:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762659.803000:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762659.803009:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565796096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762659.803029:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762659/real 1789762659] req@ffff93f844d67800 x1876699565795712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762659.803045:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565795712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762659.803053:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762659.803056:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762659.803059:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565795712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762659.803076:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565795840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762659.803091:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762659.803099:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762659/real 1789762659] req@ffff93f844d67480 x1876699565795840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762659.803107:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565795840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762659.803113:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762659.803115:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762659.803117:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565795840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762659.803122:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565795968:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762659.803133:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762659.803139:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762659/real 1789762659] req@ffff93f844d64700 x1876699565795968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762659.803167:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565795968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762659.803173:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762659.803175:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762659.803177:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565795968:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762659.803180:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565796096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762659.803192:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762659.803198:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762659/real 1789762659] req@ffff93f844d67b80 x1876699565796096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762659.803205:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565796096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762659.803210:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762659.803212:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762659.803214:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565796096:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762659.866818:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762660.890810:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762661.914760:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762662.938861:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762663.962798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762664.922746:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762664.922757:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762664.922762:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762664.922784:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2599 00000100:00080000:3.0:1789762664.922795:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762664.922803:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762664.922850:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565796224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762664.922871:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762664.922875:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762664.922877:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762664.922880:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2599 00000100:00080000:3.0:1789762664.922883:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762664.922886:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762664.922896:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762664.922898:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565796224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762664.922907:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565796352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762664.922916:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762664.922918:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762664.922919:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762664.922922:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2599 00000100:00080000:3.0:1789762664.922924:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762664.922926:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762664.922931:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762664.922941:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565796480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762664.922949:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762664.922952:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762664.922953:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762664.922955:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2599 00000100:00080000:3.0:1789762664.922957:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762664.922960:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762664.922963:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762664.922974:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565796608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789762664.922983:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762664.923049:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762664/real 1789762664] req@ffff93f844d65880 x1876699565796224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762664.923072:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565796224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762664.923086:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762664.923090:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762664.923093:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565796224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762664.923120:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565796352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762664.923140:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762664.923150:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762664/real 1789762664] req@ffff93f844d66680 x1876699565796352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762664.923159:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565796352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762664.923166:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762664.923168:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762664.923170:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565796352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762664.923174:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565796480:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762664.923187:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762664.923193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762664/real 1789762664] req@ffff93f844d67b80 x1876699565796480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762664.923201:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565796480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762664.923211:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762664.923213:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762664.923216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565796480:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762664.923219:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565796608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762664.923230:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762664.923236:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762664/real 1789762664] req@ffff93f844d64700 x1876699565796608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762664.923243:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565796608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762664.923248:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762664.923250:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762664.923252:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565796608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762664.986825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762666.010777:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762667.034819:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762668.058817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762669.082822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762670.042836:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762670.042851:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762670.042858:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762670.042872:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2605 00000100:00080000:1.0:1789762670.042887:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762670.042897:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762670.042956:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565796736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762670.043002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762670.043010:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762670.043014:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762670.043020:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2605 00000100:00080000:1.0:1789762670.043027:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762670.043035:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762670.043048:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762670.043061:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565796736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762670.043072:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565796864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762670.043088:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762670.043092:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762670.043095:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762670.043100:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2605 00000100:00080000:1.0:1789762670.043103:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762670.043108:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762670.043114:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762670.043128:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565796992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762670.043141:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762670.043147:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762670.043150:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762670.043157:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2605 00000800:00000100:0.0:1789762670.043163:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762670.043172:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762670.043184:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762670.043196:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762670.043219:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565797120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762670.043230:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762670/real 1789762670] req@ffff93f844cc9c00 x1876699565796736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762670.043266:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565796736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762670.043281:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762670.043287:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762670.043309:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565796736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762670.043344:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565796864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762670.043424:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762670.043441:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762670/real 1789762670] req@ffff93f844cca680 x1876699565796864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762670.043456:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565796864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762670.043466:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762670.043470:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762670.043473:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565796864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762670.043479:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565796992:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762670.043500:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762670.043510:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762670/real 1789762670] req@ffff93f844cc8e00 x1876699565796992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762670.043523:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565796992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762670.043531:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762670.043534:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762670.043537:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565796992:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762670.043543:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565797120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762670.043559:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762670.043569:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762670/real 1789762670] req@ffff93f844ccb100 x1876699565797120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762670.043581:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565797120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762670.043598:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762670.043602:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762670.043605:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565797120:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762670.106796:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762671.130842:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762672.154816:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762673.178777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762674.202818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762675.162865:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762675.162875:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762675.162879:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762675.162891:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2610 00000100:00080000:0.0:1789762675.162904:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762675.162913:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762675.162954:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565797248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762675.162990:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762675.162996:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762675.162998:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762675.163004:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2610 00000100:00080000:0.0:1789762675.163008:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762675.163014:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762675.163027:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762675.163043:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565797248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762675.163045:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565797376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762675.163071:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762675.163075:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762675.163076:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762675.163082:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2610 00000100:00080000:0.0:1789762675.163086:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762675.163089:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762675.163095:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762675.163108:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565797504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762675.163117:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762675.163120:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762675.163122:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762675.163125:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2610 00000100:00080000:0.0:1789762675.163131:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789762675.163132:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762675.163140:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762675.163145:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762675.163157:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565797632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762675.163189:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762675/real 1789762675] req@ffff93f850d40e00 x1876699565797248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762675.163206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565797248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762675.163217:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762675.163221:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762675.163225:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565797248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762675.163246:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565797376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762675.163265:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762675.163274:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762675/real 1789762675] req@ffff93f850d41c00 x1876699565797376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762675.163284:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565797376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762675.163294:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762675.163299:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762675.163303:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565797376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762675.163309:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565797504:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762675.163354:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762675.163373:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762675/real 1789762675] req@ffff93f850d42680 x1876699565797504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762675.163400:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565797504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762675.163412:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762675.163415:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762675.163419:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565797504:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762675.163426:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565797632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762675.163442:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762675.163457:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762675/real 1789762675] req@ffff93f850d40000 x1876699565797632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762675.163465:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565797632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762675.163471:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762675.163473:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762675.163475:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565797632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762675.226778:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762676.250773:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762677.274806:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762678.298769:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762679.322827:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762680.282738:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762680.282745:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762680.282749:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762680.282757:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2615 00000100:00080000:0.0:1789762680.282764:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762680.282770:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762680.282799:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565797760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762680.282825:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762680.282827:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762680.282829:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762680.282831:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2615 00000100:00080000:0.0:1789762680.282834:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762680.282836:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762680.282843:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762680.282852:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565797888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762680.282858:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565797760:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762680.282862:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762680.282866:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762680.282868:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762680.282870:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2615 00000100:00080000:0.0:1789762680.282872:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762680.282874:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762680.282877:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762680.282885:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565798016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762680.282892:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762680.282894:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762680.282896:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762680.282898:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2615 00000100:00080000:0.0:1789762680.282901:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762680.282903:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762680.282907:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762680.282914:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565798144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762680.282920:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762680.282976:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762680/real 1789762680] req@ffff93f850d41180 x1876699565797760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762680.282992:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565797760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762680.283004:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762680.283007:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762680.283010:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565797760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762680.283031:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565797888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762680.283048:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762680.283057:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762680/real 1789762680] req@ffff93f850d43100 x1876699565797888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762680.283066:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565797888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762680.283071:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762680.283074:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762680.283076:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565797888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762680.283080:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565798016:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762680.283090:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762680.283096:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762680/real 1789762680] req@ffff93f850d41f80 x1876699565798016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762680.283103:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565798016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762680.283109:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762680.283110:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762680.283112:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565798016:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762680.283115:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565798144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762680.283125:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762680.283131:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762680/real 1789762680] req@ffff93f850d41500 x1876699565798144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762680.283138:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565798144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762680.283143:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762680.283145:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762680.283147:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565798144:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762680.346755:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762681.370771:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762682.394781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762683.418754:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762684.442857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762685.402727:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762685.402735:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762685.402740:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762685.402750:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2620 00000100:00080000:0.0:1789762685.402759:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762685.402765:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762685.402799:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565798272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762685.402823:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762685.402826:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762685.402828:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762685.402832:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2620 00000100:00080000:0.0:1789762685.402837:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762685.402840:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762685.402848:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762685.402858:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565798400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762685.402867:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789762685.402869:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565798272:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762685.402873:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762685.402877:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762685.402880:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2620 00000100:00080000:0.0:1789762685.402884:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762685.402887:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762685.402892:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762685.402907:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565798528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762685.402919:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762685.402923:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762685.402925:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789762685.402930:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762685.402935:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2620 00000100:00080000:0.0:1789762685.402941:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762685.402945:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762685.402950:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762685.402960:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565798656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762685.402977:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762685/real 1789762685] req@ffff93f850d43480 x1876699565798272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762685.402996:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565798272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762685.403005:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762685.403008:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762685.403011:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565798272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762685.403030:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565798400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762685.403046:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762685.403054:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762685/real 1789762685] req@ffff93f850d43800 x1876699565798400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762685.403063:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565798400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762685.403073:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762685.403075:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762685.403077:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565798400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762685.403081:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565798528:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762685.403092:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762685.403097:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762685/real 1789762685] req@ffff93f850d41500 x1876699565798528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762685.403104:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565798528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762685.403110:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762685.403112:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762685.403113:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565798528:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762685.403117:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565798656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762685.403127:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762685.403133:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762685/real 1789762685] req@ffff93f850d41f80 x1876699565798656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762685.403141:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565798656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762685.403145:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762685.403147:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762685.403150:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565798656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762685.466790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762686.490787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762687.514910:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762688.538849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762689.562767:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762690.522723:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762690.522730:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762690.522734:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762690.522742:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2625 00000100:00080000:0.0:1789762690.522750:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762690.522757:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762690.522789:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565798784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762690.522812:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762690.522815:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762690.522816:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762690.522819:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2625 00000100:00080000:0.0:1789762690.522821:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762690.522824:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762690.522830:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762690.522838:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565798912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762690.522846:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762690.522848:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762690.522849:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762690.522852:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2625 00000100:00080000:0.0:1789762690.522854:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762690.522856:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762690.522860:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762690.522865:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565798784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762690.522869:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565799040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762690.522880:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762690.522882:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762690.522883:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762690.522885:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2625 00000100:00080000:0.0:1789762690.522889:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762690.522891:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762690.522894:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762690.522901:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565799168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762690.522936:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762690.522978:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762690/real 1789762690] req@ffff93f850d43100 x1876699565798784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762690.522994:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565798784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762690.523002:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762690.523006:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762690.523010:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565798784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762690.523018:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565798912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762690.523043:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762690.523057:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762690/real 1789762690] req@ffff93f850d41180 x1876699565798912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762690.523066:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565798912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762690.523072:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762690.523074:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762690.523077:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565798912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762690.523096:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565799040:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762690.523111:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762690.523118:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762690/real 1789762690] req@ffff93f850d40000 x1876699565799040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762690.523126:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565799040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762690.523131:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762690.523133:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762690.523134:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565799040:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762690.523138:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565799168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762690.523148:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762690.523153:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762690/real 1789762690] req@ffff93f850d42680 x1876699565799168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762690.523161:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565799168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762690.523165:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762690.523167:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762690.523169:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565799168:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762690.586832:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762691.610787:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762692.634796:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762693.658779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762694.682780:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762695.643137:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762695.643148:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762695.643153:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762695.643178:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2630 00000100:00080000:2.0:1789762695.643190:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762695.643198:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762695.643237:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565799296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762695.643269:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762695.643272:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762695.643274:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762695.643277:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2630 00000100:00080000:2.0:1789762695.643280:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762695.643284:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762695.643292:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762695.643303:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565799424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762695.643311:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789762695.643316:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565799296:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762695.643316:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762695.643318:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762695.643325:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2630 00000100:00080000:2.0:1789762695.643328:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762695.643330:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762695.643333:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762695.643342:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565799552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762695.643353:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762695.643357:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762695.643359:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762695.643364:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2630 00000100:00080000:2.0:1789762695.643369:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762695.643373:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789762695.643378:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1789762695.643378:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762695.643394:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565799680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762695.643423:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762695/real 1789762695] req@ffff93f8517c2300 x1876699565799296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762695.643442:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565799296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762695.643453:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762695.643466:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762695.643471:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565799296:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762695.643496:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565799424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762695.643531:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762695.643546:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762695/real 1789762695] req@ffff93f8517c1f80 x1876699565799424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762695.643559:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565799424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762695.643568:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762695.643571:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762695.643575:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565799424:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762695.643582:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565799552:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762695.643603:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762695.643613:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762695/real 1789762695] req@ffff93f8517c3b80 x1876699565799552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762695.643621:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565799552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762695.643627:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762695.643629:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762695.643631:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565799552:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762695.643635:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565799680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762695.643672:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762695.643687:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762695/real 1789762695] req@ffff93f8517c0000 x1876699565799680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762695.643713:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565799680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762695.643722:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762695.643726:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762695.643729:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565799680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762695.706766:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762696.730770:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762697.754803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762698.778797:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762699.802774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762700.762759:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762700.762770:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762700.762774:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762700.762785:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2635 00000100:00080000:1.0:1789762700.762793:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762700.762801:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762700.762838:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565799808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762700.762863:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762700.762867:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762700.762869:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762700.762875:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2635 00000100:00080000:1.0:1789762700.762879:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762700.762884:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762700.762888:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565799808:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762700.762896:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762700.762915:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565799936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762700.762924:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762700.762926:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762700.762928:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762700.762931:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2635 00000100:00080000:1.0:1789762700.762937:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762700.762941:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762700.762945:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789762700.762947:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762700.762957:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565800064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762700.762966:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762700.762968:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762700.762969:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762700.762971:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2635 00000100:00080000:1.0:1789762700.762974:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762700.762976:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762700.762979:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762700.762989:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762700/real 1789762700] req@ffff93f844cc9500 x1876699565799808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762700.762990:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565800192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762700.763025:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565799808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762700.763034:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762700.763038:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762700.763042:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565799808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762700.763059:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565799936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762700.763074:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762700.763082:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762700/real 1789762700] req@ffff93f844ccad80 x1876699565799936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762700.763091:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565799936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762700.763096:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762700.763099:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762700.763101:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565799936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762700.763104:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565800064:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762700.763149:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762700.763159:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762700/real 1789762700] req@ffff93f844ccb100 x1876699565800064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762700.763168:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565800064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762700.763178:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762700.763180:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762700.763182:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565800064:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762700.763186:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565800192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762700.763199:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762700.763208:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762700/real 1789762700] req@ffff93f844cc8e00 x1876699565800192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762700.763215:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565800192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762700.763221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762700.763222:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762700.763224:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565800192:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762700.826764:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762701.850769:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762702.874794:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762703.898821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762704.922786:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762705.882741:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762705.882750:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762705.882753:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762705.882762:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2640 00000100:00080000:0.0:1789762705.882770:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762705.882777:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762705.882809:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565800320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762705.882833:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762705.882836:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762705.882837:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762705.882840:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2640 00000100:00080000:0.0:1789762705.882843:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762705.882847:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762705.882854:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762705.882865:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565800448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762705.882865:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565800320:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762705.882880:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762705.882882:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762705.882883:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762705.882886:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2640 00000100:00080000:0.0:1789762705.882888:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762705.882891:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762705.882894:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762705.882902:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565800576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762705.882910:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762705.882912:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762705.882914:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762705.882917:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2640 00000100:00080000:0.0:1789762705.882919:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762705.882922:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762705.882926:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789762705.882930:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762705.882938:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565800704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762705.882971:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762705/real 1789762705] req@ffff93f850d41c00 x1876699565800320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762705.882985:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565800320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762705.882993:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762705.882996:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762705.882999:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565800320:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762705.883016:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565800448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762705.883031:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762705.883039:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762705/real 1789762705] req@ffff93f850d40e00 x1876699565800448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762705.883047:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565800448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762705.883053:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762705.883055:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762705.883057:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565800448:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762705.883061:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565800576:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762705.883071:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762705.883077:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762705/real 1789762705] req@ffff93f850d42300 x1876699565800576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762705.883085:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565800576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762705.883103:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762705.883106:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762705.883108:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565800576:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762705.883112:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565800704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762705.883127:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762705.883134:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762705/real 1789762705] req@ffff93f850d40380 x1876699565800704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762705.883141:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565800704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762705.883146:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762705.883148:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762705.883150:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565800704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762705.946787:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762706.970758:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762707.994763:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762709.018767:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762710.042805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762711.002756:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762711.002767:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762711.002772:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762711.002783:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2645 00000100:00080000:0.0:1789762711.002794:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762711.002803:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762711.002850:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565800832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762711.002887:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762711.002892:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762711.002894:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762711.002899:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2645 00000100:00080000:0.0:1789762711.002905:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762711.002910:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762711.002922:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762711.002924:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565800832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762711.002939:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565800960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762711.002953:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762711.002957:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762711.002960:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762711.002965:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2645 00000100:00080000:0.0:1789762711.002971:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762711.002976:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762711.002982:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762711.002995:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565801088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789762711.003008:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762711.003009:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762711.003015:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762711.003018:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762711.003024:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2645 00000100:00080000:0.0:1789762711.003034:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762711.003039:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762711.003044:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762711.003073:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762711/real 1789762711] req@ffff93f850d40700 x1876699565800832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762711.003074:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565801216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762711.003121:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565800832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762711.003135:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762711.003140:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762711.003145:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565800832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762711.003172:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565800960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762711.003197:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762711.003212:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762711/real 1789762711] req@ffff93f850d43b80 x1876699565800960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762711.003227:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565800960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762711.003236:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762711.003240:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762711.003245:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565800960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762711.003252:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565801088:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762711.003274:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762711.003289:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762711/real 1789762711] req@ffff93f850d40380 x1876699565801088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762711.003302:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565801088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762711.003314:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762711.003319:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762711.003324:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565801088:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762711.003332:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565801216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762711.003375:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762711.003389:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762711/real 1789762711] req@ffff93f850d42300 x1876699565801216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762711.003403:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565801216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762711.003412:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762711.003416:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762711.003419:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565801216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762711.066762:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762712.090872:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762713.114768:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762714.138783:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762715.162779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762716.122722:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762716.122733:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762716.122738:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762716.122749:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2651 00000100:00080000:0.0:1789762716.122759:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762716.122766:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762716.122807:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565801344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762716.122830:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762716.122835:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762716.122837:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762716.122840:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2651 00000100:00080000:0.0:1789762716.122842:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762716.122845:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762716.122854:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762716.122862:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565801472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762716.122869:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762716.122871:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762716.122873:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762716.122875:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2651 00000100:00080000:0.0:1789762716.122877:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762716.122880:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762716.122883:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762716.122890:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565801600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762716.122898:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762716.122902:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762716.122904:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789762716.122904:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565801344:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762716.122912:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2651 00000100:00080000:0.0:1789762716.122915:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762716.122917:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762716.122921:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762716.122930:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565801728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762716.123027:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565801472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762716.123063:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565801600:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762716.123080:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565801728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762716.186783:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762717.210820:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762718.234859:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1789762719.195119:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1789762719.195190:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1789762719.195215:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789762719.195291:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789762719.195313:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789762719.195338:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789762719.195384:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762716/real 1789762719] req@ffff93f850d42680 x1876699565801600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762719.195439:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565801600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762719.195465:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762719.195475:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762719.195485:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565801600:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762719.195528:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762716/real 1789762719] req@ffff93f850d40000 x1876699565801728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762719.195557:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565801728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762719.195575:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762719.195583:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762719.195590:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565801728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762719.195604:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762716/real 1789762719] req@ffff93f850d40e00 x1876699565801344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762719.195630:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565801344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762719.195695:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762719.195704:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762719.195712:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565801344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762719.195727:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762716/real 1789762719] req@ffff93f850d41c00 x1876699565801472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762719.195755:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565801472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762719.195773:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762719.195780:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762719.195787:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565801472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762719.258844:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762720.282871:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762721.242792:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762721.242801:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762721.242805:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762721.242817:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2656 00000100:00080000:0.0:1789762721.242829:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762721.242836:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762721.242864:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565801856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762721.242884:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762721.242886:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762721.242888:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762721.242891:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2656 00000100:00080000:0.0:1789762721.242893:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762721.242895:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762721.242902:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762721.242912:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565801984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762721.242922:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762721.242926:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762721.242928:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762721.242932:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2656 00000100:00080000:0.0:1789762721.242936:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762721.242940:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762721.242945:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762721.242957:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565802112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762721.242968:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762721.242985:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762721.242988:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789762721.242993:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565801856:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762721.242996:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2656 00000100:00080000:0.0:1789762721.243004:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762721.243008:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762721.243014:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762721.243028:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565802240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789762721.243068:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762721.243125:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762721/real 1789762721] req@ffff93f850d41180 x1876699565801856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762721.243145:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565801856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762721.243157:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762721.243161:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762721.243165:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565801856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762721.243173:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565801984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762721.243206:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762721.243220:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762721/real 1789762721] req@ffff93f850d43100 x1876699565801984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762721.243234:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565801984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762721.243242:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762721.243246:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762721.243250:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565801984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762721.243255:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565802112:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762721.243279:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762721.243292:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762721/real 1789762721] req@ffff93f850d41f80 x1876699565802112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762721.243303:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565802112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762721.243319:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762721.243322:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762721.243326:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565802112:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762721.243354:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565802240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762721.243377:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762721.243390:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762721/real 1789762721] req@ffff93f850d41500 x1876699565802240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762721.243402:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565802240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762721.243410:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762721.243414:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762721.243417:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565802240:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762721.306802:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762722.330803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762723.354792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762724.378789:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762725.402778:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762726.362776:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762726.362787:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762726.362792:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762726.362803:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2661 00000100:00080000:1.0:1789762726.362815:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762726.362821:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762726.362857:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565802368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762726.362881:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762726.362886:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762726.362889:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762726.362894:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2661 00000100:00080000:1.0:1789762726.362898:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762726.362903:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762726.362912:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762726.362928:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565802368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762726.362929:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565802496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762726.362966:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762726.362969:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762726.362970:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762726.362974:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2661 00000100:00080000:1.0:1789762726.362977:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762726.362981:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762726.362986:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762726.362996:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565802624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762726.363004:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762726.363007:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762726.363008:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762726.363011:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2661 00000100:00080000:1.0:1789762726.363015:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762726.363017:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762726.363021:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762726.363029:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565802752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762726.363061:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762726.363136:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762726/real 1789762726] req@ffff93f844cca680 x1876699565802368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762726.363160:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565802368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762726.363174:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762726.363179:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762726.363183:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565802368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762726.363217:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565802496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762726.363247:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762726.363264:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762726/real 1789762726] req@ffff93f844cc9c00 x1876699565802496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762726.363278:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565802496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762726.363287:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762726.363292:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762726.363296:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565802496:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762726.363303:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565802624:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762726.363327:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762726.363340:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762726/real 1789762726] req@ffff93f844ccaa00 x1876699565802624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762726.363354:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565802624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762726.363363:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762726.363367:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762726.363371:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565802624:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762726.363378:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565802752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762726.363404:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762726.363418:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762726/real 1789762726] req@ffff93f844ccb480 x1876699565802752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762726.363431:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565802752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762726.363440:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762726.363443:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762726.363447:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565802752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762726.426745:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762727.450791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762728.474751:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762729.498768:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762730.522799:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762731.482766:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762731.482774:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762731.482777:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762731.482785:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2666 00000100:00080000:0.0:1789762731.482793:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762731.482799:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762731.482831:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565802880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762731.482858:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762731.482860:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762731.482861:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762731.482865:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2666 00000100:00080000:0.0:1789762731.482867:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762731.482870:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762731.482877:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762731.482885:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565803008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762731.482891:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565802880:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762731.482918:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762731.482920:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762731.482922:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762731.482925:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2666 00000100:00080000:0.0:1789762731.482929:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762731.482931:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762731.482934:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762731.482944:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565803136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762731.482951:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762731.482953:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762731.482955:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762731.482957:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2666 00000100:00080000:0.0:1789762731.482960:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762731.482962:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762731.482965:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789762731.482969:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762731.482974:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565803264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762731.483010:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762731/real 1789762731] req@ffff93f850d43800 x1876699565802880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762731.483025:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565802880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762731.483033:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762731.483036:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762731.483039:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565802880:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762731.483056:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565803008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762731.483071:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762731.483079:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762731/real 1789762731] req@ffff93f850d43480 x1876699565803008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762731.483087:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565803008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762731.483092:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762731.483094:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762731.483096:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565803008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762731.483100:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565803136:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762731.483111:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762731.483117:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762731/real 1789762731] req@ffff93f850d41500 x1876699565803136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762731.483125:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565803136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762731.483130:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762731.483131:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762731.483133:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565803136:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762731.483136:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565803264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762731.483146:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762731.483172:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762731/real 1789762731] req@ffff93f850d41f80 x1876699565803264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762731.483186:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565803264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762731.483194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762731.483197:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762731.483200:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565803264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762731.546819:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762732.570778:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762733.594765:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762734.618794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762735.642792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762736.602769:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762736.602790:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762736.602795:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762736.602806:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2671 00000100:00080000:1.0:1789762736.602816:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762736.602824:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762736.602872:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565803392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762736.602910:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762736.602914:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762736.602917:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762736.602922:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2671 00000100:00080000:1.0:1789762736.602928:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762736.602933:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762736.602938:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565803392:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762736.602944:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762736.602962:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565803520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762736.602975:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762736.602980:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762736.602982:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762736.602987:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2671 00000100:00080000:1.0:1789762736.602992:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789762736.602996:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762736.602999:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762736.603007:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762736.603021:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565803648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762736.603032:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762736.603035:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762736.603038:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789762736.603042:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762736/real 1789762736] req@ffff93f844ccb800 x1876699565803392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762736.603046:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2671 00000100:00080000:1.0:1789762736.603054:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762736.603060:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762736.603064:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565803392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1789762736.603068:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00080000:2.0:1789762736.603077:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762736.603081:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762736.603088:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565803392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762736.603089:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565803776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762736.603110:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565803520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762736.603128:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762736.603137:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762736/real 1789762736] req@ffff93f844cc8700 x1876699565803520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762736.603145:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565803520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762736.603150:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762736.603152:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762736.603154:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565803520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762736.603158:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565803648:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762736.603169:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762736.603175:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762736/real 1789762736] req@ffff93f844ccbb80 x1876699565803648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762736.603182:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565803648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762736.603187:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762736.603189:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762736.603191:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565803648:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762736.603194:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565803776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762736.603204:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762736.603209:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762736/real 1789762736] req@ffff93f844cc8000 x1876699565803776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762736.603216:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565803776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762736.603221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762736.603223:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762736.603225:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565803776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762736.666776:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762737.690816:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762738.714804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762739.738826:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762740.762858:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762741.722744:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762741.722753:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762741.722756:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762741.722764:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2676 00000100:00080000:0.0:1789762741.722773:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762741.722779:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762741.722810:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565803904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762741.722843:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762741.722846:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762741.722847:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762741.722850:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2676 00000100:00080000:0.0:1789762741.722853:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762741.722857:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762741.722864:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762741.722878:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565804032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762741.722879:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565803904:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762741.722893:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762741.722896:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762741.722897:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762741.722900:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2676 00000100:00080000:0.0:1789762741.722902:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762741.722904:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762741.722907:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762741.722914:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565804160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762741.722920:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762741.722923:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762741.722925:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762741.722927:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2676 00000100:00080000:0.0:1789762741.722930:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762741.722932:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762741.722934:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762741.722941:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565804288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762741.722951:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762741.722998:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762741/real 1789762741] req@ffff93f850d43100 x1876699565803904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762741.723019:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565803904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762741.723031:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762741.723036:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762741.723039:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565803904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762741.723060:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565804032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762741.723079:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762741.723089:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762741/real 1789762741] req@ffff93f850d41180 x1876699565804032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762741.723113:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565804032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762741.723119:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762741.723121:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762741.723123:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565804032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762741.723127:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565804160:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762741.723142:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762741.723149:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762741/real 1789762741] req@ffff93f850d41c00 x1876699565804160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762741.723157:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565804160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762741.723166:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762741.723168:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762741.723170:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565804160:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762741.723174:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565804288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762741.723184:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762741.723190:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762741/real 1789762741] req@ffff93f850d40e00 x1876699565804288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762741.723197:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565804288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762741.723202:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762741.723204:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762741.723206:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565804288:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762741.786808:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762742.810861:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762743.834820:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762744.858855:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762745.882764:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762746.842740:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762746.842747:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762746.842751:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762746.842759:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2681 00000100:00080000:0.0:1789762746.842767:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762746.842773:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762746.842822:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565804416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762746.842841:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762746.842844:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762746.842847:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762746.842850:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2681 00000100:00080000:0.0:1789762746.842853:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762746.842856:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762746.842864:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762746.842876:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565804544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762746.842884:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762746.842886:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762746.842887:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762746.842906:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2681 00000100:00080000:0.0:1789762746.842909:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762746.842916:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762746.842916:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565804416:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1789762746.842925:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762746.842937:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565804672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762746.842946:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762746.842949:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762746.842951:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762746.842954:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2681 00000100:00080000:0.0:1789762746.842959:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762746.842962:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762746.842966:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762746.842974:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565804800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762746.842995:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762746.843051:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762746/real 1789762746] req@ffff93f850d40000 x1876699565804416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762746.843066:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565804416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762746.843075:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762746.843078:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762746.843081:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565804416:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762746.843086:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565804544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762746.843103:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762746.843111:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762746/real 1789762746] req@ffff93f850d42680 x1876699565804544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762746.843119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565804544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762746.843124:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762746.843126:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762746.843128:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565804544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762746.843152:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565804672:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762746.843164:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762746.843170:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762746/real 1789762746] req@ffff93f850d42300 x1876699565804672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762746.843178:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565804672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762746.843183:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762746.843185:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762746.843187:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565804672:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762746.843191:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565804800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762746.843201:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762746.843207:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762746/real 1789762746] req@ffff93f850d40380 x1876699565804800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762746.843215:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565804800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762746.843220:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762746.843222:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762746.843224:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565804800:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762746.906771:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762747.930745:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762748.954785:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762749.978817:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762751.002849:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762751.962734:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762751.962754:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762751.962759:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762751.962768:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2686 00000100:00080000:2.0:1789762751.962776:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762751.962782:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762751.962811:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565804928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762751.962839:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762751.962842:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762751.962844:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762751.962848:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2686 00000100:00080000:2.0:1789762751.962850:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762751.962853:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762751.962861:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762751.962868:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565804928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762751.962872:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565805056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762751.962887:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762751.962889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762751.962891:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762751.962894:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2686 00000100:00080000:2.0:1789762751.962896:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762751.962899:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762751.962902:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762751.962909:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565805184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762751.962916:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762751.962919:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762751.962920:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762751.962923:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2686 00000800:00000100:0.0:1789762751.962928:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762751.962929:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762751.962934:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762751.962937:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762751.962946:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565805312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762751.962971:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762751/real 1789762751] req@ffff93f8517c0000 x1876699565804928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762751.962995:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565804928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762751.963007:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762751.963010:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762751.963013:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565804928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762751.963033:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565805056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762751.963057:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762751.963072:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762751/real 1789762751] req@ffff93f8517c3b80 x1876699565805056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762751.963087:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565805056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762751.963095:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762751.963098:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762751.963100:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565805056:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762751.963105:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565805184:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762751.963121:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762751.963129:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762751/real 1789762751] req@ffff93f8517c1f80 x1876699565805184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762751.963139:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565805184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762751.963154:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762751.963158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762751.963162:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565805184:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762751.963168:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565805312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762751.963188:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762751.963197:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762751/real 1789762751] req@ffff93f8517c2300 x1876699565805312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762751.963206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565805312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762751.963211:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762751.963213:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762751.963215:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565805312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762752.026764:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762753.050803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762754.074845:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762755.098853:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762756.122835:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762757.082775:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762757.082786:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762757.082791:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762757.082802:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2692 00000100:00080000:1.0:1789762757.082813:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762757.082821:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762757.082867:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565805440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762757.082895:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762757.082898:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762757.082899:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762757.082902:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2692 00000100:00080000:1.0:1789762757.082905:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762757.082907:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762757.082916:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762757.082927:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565805568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762757.082940:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762757.082943:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762757.082944:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762757.082950:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2692 00000100:00080000:1.0:1789762757.082958:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789762757.082960:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565805440:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762757.082961:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762757.082966:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762757.082978:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565805696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762757.082986:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762757.082989:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762757.082991:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762757.082994:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2692 00000100:00080000:1.0:1789762757.082996:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762757.082999:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762757.083002:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762757.083011:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565805824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762757.083051:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762757.083107:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762757/real 1789762757] req@ffff93f844cc8380 x1876699565805440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762757.083128:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565805440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762757.083140:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762757.083144:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762757.083149:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565805440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762757.083174:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565805568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762757.083196:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762757.083209:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762757/real 1789762757] req@ffff93f844cca300 x1876699565805568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762757.083222:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565805568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762757.083230:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762757.083234:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762757.083237:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565805568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762757.083244:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565805696:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762757.083263:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762757.083273:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762757/real 1789762757] req@ffff93f844cc9f80 x1876699565805696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762757.083286:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565805696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762757.083294:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762757.083298:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762757.083301:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565805696:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762757.083307:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565805824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762757.083324:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762757.083335:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762757/real 1789762757] req@ffff93f844cc9880 x1876699565805824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762757.083347:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565805824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762757.083355:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762757.083358:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762757.083361:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565805824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762757.146833:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762758.170804:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762759.194787:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762760.218803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762761.242777:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762762.202983:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762762.202994:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762762.202999:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762762.203008:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2697 00000100:00080000:1.0:1789762762.203019:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762762.203027:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762762.203072:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565805952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762762.203104:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762762.203109:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762762.203111:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762762.203116:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2697 00000100:00080000:1.0:1789762762.203120:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762762.203126:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762762.203136:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762762.203153:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565806080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762762.203164:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762762.203167:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762762.203169:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762762.203174:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2697 00000100:00080000:1.0:1789762762.203178:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789762762.203182:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565805952:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762762.203186:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762762.203197:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762762.203211:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565806208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762762.203234:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762762.203238:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762762.203241:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762762.203245:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2697 00000100:00080000:1.0:1789762762.203250:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762762.203255:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762762.203261:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789762762.203268:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762762.203283:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565806336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762762.203324:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762762/real 1789762762] req@ffff93f844cc9180 x1876699565805952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762762.203350:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565805952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762762.203361:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762762.203366:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762762.203370:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565805952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762762.203378:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565806080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762762.203401:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762762.203414:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762762/real 1789762762] req@ffff93f844cc8a80 x1876699565806080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762762.203427:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565806080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762762.203447:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762762.203450:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762762.203454:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565806080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762762.203479:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565806208:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762762.203612:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762762.203626:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762762/real 1789762762] req@ffff93f844cc9880 x1876699565806208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762762.203640:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565806208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762762.203690:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762762.203695:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762762.203707:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565806208:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762762.203714:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565806336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762762.203738:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762762.203750:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762762/real 1789762762] req@ffff93f844cc9f80 x1876699565806336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762762.203763:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565806336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762762.203771:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762762.203775:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762762.203778:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565806336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762762.266836:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762763.290770:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762764.314793:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762765.338829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762766.362776:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762767.322729:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762767.322738:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762767.322741:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762767.322750:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2702 00000100:00080000:1.0:1789762767.322760:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762767.322767:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762767.322800:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565806464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762767.322823:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762767.322826:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762767.322828:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762767.322832:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2702 00000100:00080000:1.0:1789762767.322835:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762767.322838:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762767.322845:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762767.322853:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565806464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762767.322858:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565806592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762767.322868:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762767.322870:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762767.322871:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762767.322874:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2702 00000100:00080000:1.0:1789762767.322876:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762767.322878:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762767.322881:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762767.322889:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565806720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762767.322896:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762767.322899:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762767.322907:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789762767.322912:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762767.322913:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2702 00000100:00080000:1.0:1789762767.322919:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762767.322922:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762767.322926:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762767.322935:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565806848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762767.322961:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762767/real 1789762767] req@ffff93f844cca300 x1876699565806464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762767.322975:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565806464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762767.322985:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762767.322988:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762767.322992:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565806464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762767.323011:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565806592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762767.323027:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762767.323036:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762767/real 1789762767] req@ffff93f844cc8380 x1876699565806592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762767.323051:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565806592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762767.323057:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762767.323060:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762767.323062:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565806592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762767.323066:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565806720:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762767.323078:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762767.323085:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762767/real 1789762767] req@ffff93f844cc8000 x1876699565806720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762767.323096:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565806720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762767.323102:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762767.323104:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762767.323106:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565806720:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762767.323109:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565806848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762767.323120:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762767.323126:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762767/real 1789762767] req@ffff93f844ccbb80 x1876699565806848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762767.323134:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565806848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762767.323139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762767.323141:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762767.323142:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565806848:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762767.386784:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762768.410803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762769.434764:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762770.458817:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762771.482805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762772.442741:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762772.442752:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762772.442757:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762772.442769:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2707 00000100:00080000:1.0:1789762772.442780:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762772.442788:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762772.442826:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565806976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762772.442853:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762772.442857:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762772.442859:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762772.442863:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2707 00000100:00080000:1.0:1789762772.442867:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762772.442871:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762772.442880:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762772.442892:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565806976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762772.442896:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565807104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762772.442918:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762772.442923:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762772.442926:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762772.442932:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2707 00000100:00080000:1.0:1789762772.442937:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762772.442943:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762772.442950:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762772.442967:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565807232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762772.442971:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762772.442984:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762772.442988:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762772.442990:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762772.442995:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2707 00000100:00080000:1.0:1789762772.443001:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762772.443006:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762772.443013:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762772.443026:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762772/real 1789762772] req@ffff93f844cc8700 x1876699565806976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762772.443035:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565807360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762772.443055:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565806976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762772.443069:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762772.443074:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762772.443079:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565806976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762772.443103:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565807104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762772.443126:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762772.443139:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762772/real 1789762772] req@ffff93f844ccb800 x1876699565807104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762772.443152:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565807104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762772.443167:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762772.443171:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762772.443175:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565807104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762772.443181:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565807232:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762772.443203:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762772.443215:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762772/real 1789762772] req@ffff93f844ccb480 x1876699565807232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762772.443229:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565807232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762772.443239:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762772.443242:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762772.443246:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565807232:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762772.443252:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565807360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762772.443272:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762772.443284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762772/real 1789762772] req@ffff93f844ccaa00 x1876699565807360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762772.443296:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565807360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762772.443305:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762772.443309:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762772.443312:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565807360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762772.506803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762773.530829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762774.554768:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762775.578774:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762776.602833:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762777.562761:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762777.562772:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762777.562776:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762777.562789:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2712 00000100:00080000:0.0:1789762777.562801:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762777.562809:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762777.562870:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565807488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762777.562910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762777.562914:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762777.562916:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762777.562921:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2712 00000100:00080000:0.0:1789762777.562924:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762777.562928:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762777.562953:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762777.562963:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565807616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762777.562975:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565807488:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762777.562996:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762777.563000:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762777.563001:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762777.563006:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2712 00000100:00080000:0.0:1789762777.563009:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762777.563013:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762777.563017:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762777.563029:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565807744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762777.563037:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762777.563041:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762777.563043:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762777.563046:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2712 00000100:00080000:0.0:1789762777.563048:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762777.563051:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762777.563056:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789762777.563057:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762777.563071:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565807872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762777.563112:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762777/real 1789762777] req@ffff93f850d43b80 x1876699565807488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762777.563129:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565807488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762777.563139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762777.563142:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762777.563145:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565807488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762777.563165:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565807616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762777.563181:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762777.563189:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762777/real 1789762777] req@ffff93f850d40700 x1876699565807616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762777.563206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565807616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762777.563212:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762777.563214:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762777.563216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565807616:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762777.563221:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565807744:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762777.563234:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762777.563241:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762777/real 1789762777] req@ffff93f850d40380 x1876699565807744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762777.563249:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565807744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762777.563255:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762777.563257:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762777.563258:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565807744:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762777.563262:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565807872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762777.563274:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762777.563280:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762777/real 1789762777] req@ffff93f850d42300 x1876699565807872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762777.563288:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565807872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762777.563293:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762777.563294:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762777.563296:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565807872:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762777.626830:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762778.650776:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762779.674785:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762780.698774:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762781.722809:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762782.682738:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762782.682746:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762782.682750:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762782.682758:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2717 00000100:00080000:0.0:1789762782.682768:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762782.682775:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762782.682805:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565808000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762782.682840:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762782.682845:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762782.682848:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762782.682854:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2717 00000100:00080000:0.0:1789762782.682859:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762782.682864:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762782.682873:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565808000:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1789762782.682875:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762782.682897:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565808128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762782.682911:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762782.682913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762782.682915:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762782.682918:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2717 00000100:00080000:0.0:1789762782.682924:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762782.682928:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762782.682932:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762782.682950:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565808256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762782.682959:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762782.682963:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762782.682964:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762782.682968:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2717 00000100:00080000:0.0:1789762782.682971:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762782.682976:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762782.682977:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565808128:192.168.203.147@tcp:38:kworker.0 00000008:00100000:0.0:1789762782.682984:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762782.682996:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565808384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762782.683020:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565808256:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762782.683050:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565808384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762782.746834:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762783.770798:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762784.794805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789762785.754884:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789762785.754936:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789762785.754952:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762785.754992:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762785.755008:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762785.755020:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789762785.755030:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762782/real 1789762785] req@ffff93f850d40000 x1876699565808128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762785.755052:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565808128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762785.755061:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762785.755065:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762785.755068:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565808128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762785.755073:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762782/real 1789762785] req@ffff93f850d40e00 x1876699565808256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762785.755086:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565808256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762785.755092:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762785.755094:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762785.755096:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565808256:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762785.755100:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762782/real 1789762785] req@ffff93f850d42680 x1876699565808000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762785.755107:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565808000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762785.755111:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762785.755114:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762785.755118:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565808000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762785.755149:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762782/real 1789762785] req@ffff93f850d41c00 x1876699565808384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762785.755162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565808384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762785.755169:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762785.755172:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762785.755175:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565808384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762785.818854:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762786.842800:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762787.802773:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762787.802783:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762787.802787:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762787.802796:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2722 00000100:00080000:1.0:1789762787.802804:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762787.802810:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762787.802840:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565808512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762787.802863:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762787.802866:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762787.802868:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762787.802871:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2722 00000100:00080000:1.0:1789762787.802874:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762787.802877:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762787.802885:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762787.802894:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565808640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762787.802903:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565808512:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762787.802905:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762787.802912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762787.802915:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762787.802918:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2722 00000100:00080000:1.0:1789762787.802921:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762787.802922:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762787.802925:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762787.802932:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565808768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762787.802940:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762787.802943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762787.802944:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762787.802946:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2722 00000100:00080000:1.0:1789762787.802948:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762787.802951:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762787.802954:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762787.802960:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565808896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762787.802982:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762787.803042:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762787/real 1789762787] req@ffff93f844cc9c00 x1876699565808512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762787.803064:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565808512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762787.803075:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762787.803080:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762787.803084:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565808512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762787.803111:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565808640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762787.803136:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762787.803149:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762787/real 1789762787] req@ffff93f844cca680 x1876699565808640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762787.803160:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565808640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762787.803167:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762787.803170:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762787.803173:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565808640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762787.803179:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565808768:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762787.803197:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762787.803207:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762787/real 1789762787] req@ffff93f844cc8e00 x1876699565808768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762787.803219:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565808768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762787.803227:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762787.803231:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762787.803234:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565808768:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762787.803240:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565808896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762787.803260:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762787.803272:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762787/real 1789762787] req@ffff93f844ccb100 x1876699565808896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762787.803284:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565808896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762787.803293:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762787.803302:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762787.803307:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565808896:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762787.866801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762788.890767:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762789.914735:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762790.938791:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762791.962768:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762792.922778:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762792.922787:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762792.922790:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762792.922799:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2727 00000100:00080000:2.0:1789762792.922807:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762792.922813:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762792.922851:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565809024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762792.922875:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762792.922878:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762792.922880:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762792.922882:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2727 00000100:00080000:2.0:1789762792.922885:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762792.922889:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762792.922896:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762792.922905:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565809152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762792.922914:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762792.922918:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762792.922919:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762792.922923:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2727 00000100:00100000:1.0:1789762792.922924:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565809024:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762792.922925:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762792.922932:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762792.922936:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762792.922944:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565809280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762792.922951:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762792.922954:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762792.922955:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762792.922958:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2727 00000100:00080000:2.0:1789762792.922962:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762792.922965:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762792.922967:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762792.922974:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565809408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789762792.922984:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762792.923024:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762792/real 1789762792] req@ffff93f8517c0a80 x1876699565809024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762792.923040:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565809024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762792.923048:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762792.923052:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762792.923054:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565809024:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762792.923073:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565809152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762792.923088:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762792.923096:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762792/real 1789762792] req@ffff93f8517c2680 x1876699565809152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762792.923105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565809152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762792.923111:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762792.923113:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762792.923115:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565809152:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762792.923119:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565809280:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762792.923130:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762792.923136:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762792/real 1789762792] req@ffff93f8517c1500 x1876699565809280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762792.923143:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565809280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762792.923148:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762792.923150:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762792.923151:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565809280:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762792.923155:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565809408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762792.923165:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762792.923178:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762792/real 1789762792] req@ffff93f8517c0380 x1876699565809408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762792.923185:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565809408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762792.923194:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762792.923197:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762792.923199:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565809408:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762792.986797:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762794.010800:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762795.034822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762796.058762:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762797.082773:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762798.042748:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762798.042756:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762798.042760:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762798.042769:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2733 00000100:00080000:2.0:1789762798.042778:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762798.042785:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762798.042816:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565809536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762798.042836:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762798.042839:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762798.042840:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762798.042843:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2733 00000100:00080000:2.0:1789762798.042846:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762798.042849:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762798.042857:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762798.042863:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565809536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762798.042869:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565809664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762798.042884:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762798.042887:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762798.042888:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762798.042891:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2733 00000100:00080000:2.0:1789762798.042894:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762798.042896:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762798.042900:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762798.042907:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565809792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762798.042914:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762798.042917:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762798.042919:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762798.042921:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2733 00000100:00080000:2.0:1789762798.042923:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762798.042928:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1789762798.042930:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1789762798.042933:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762798.042942:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565809920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762798.042966:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762798/real 1789762798] req@ffff93f8517c1180 x1876699565809536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762798.042982:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565809536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762798.042989:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762798.042992:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762798.042995:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565809536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762798.043012:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565809664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762798.043027:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762798.043035:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762798/real 1789762798] req@ffff93f8517c3100 x1876699565809664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762798.043043:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565809664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762798.043048:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762798.043050:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762798.043052:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565809664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762798.043057:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565809792:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762798.043068:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762798.043074:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762798/real 1789762798] req@ffff93f8517c2a00 x1876699565809792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762798.043081:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565809792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762798.043086:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762798.043088:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762798.043090:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565809792:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762798.043093:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565809920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762798.043103:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762798.043109:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762798/real 1789762798] req@ffff93f8517c2d80 x1876699565809920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762798.043116:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565809920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762798.043121:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762798.043122:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762798.043125:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565809920:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762798.106779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762799.130790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762800.154788:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762801.178820:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762802.202821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762803.162746:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762803.162755:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762803.162758:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762803.162768:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2738 00000100:00080000:1.0:1789762803.162776:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762803.162789:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762803.162826:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565810048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762803.162851:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762803.162855:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762803.162856:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762803.162861:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2738 00000100:00080000:1.0:1789762803.162866:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762803.162871:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762803.162881:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762803.162887:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565810048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762803.162899:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565810176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762803.162909:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762803.162912:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762803.162913:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762803.162916:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2738 00000100:00080000:1.0:1789762803.162919:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762803.162922:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762803.162926:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762803.162935:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565810304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762803.162947:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762803.162950:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762803.162951:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1789762803.162956:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762803.162965:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2738 00000100:00080000:1.0:1789762803.162969:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762803.162972:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762803.162976:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762803.162984:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565810432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762803.163005:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762803/real 1789762803] req@ffff93f844ccad80 x1876699565810048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762803.163025:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565810048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762803.163034:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762803.163037:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762803.163040:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565810048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762803.163060:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565810176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762803.163075:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762803.163083:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762803/real 1789762803] req@ffff93f844cc9500 x1876699565810176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762803.163092:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565810176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762803.163098:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762803.163100:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762803.163102:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565810176:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762803.163105:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565810304:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762803.163116:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762803.163122:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762803/real 1789762803] req@ffff93f844ccb100 x1876699565810304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762803.163129:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565810304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762803.163134:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762803.163135:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762803.163137:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565810304:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762803.163140:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565810432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762803.163149:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762803.163155:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762803/real 1789762803] req@ffff93f844cc8e00 x1876699565810432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762803.163162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565810432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762803.163171:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762803.163173:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762803.163175:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565810432:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762803.226816:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762804.250779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762805.274806:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762806.298766:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762807.322865:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762808.282700:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762808.282707:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762808.282711:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762808.282720:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2743 00000100:00080000:1.0:1789762808.282730:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762808.282737:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762808.282774:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565810560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762808.282805:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762808.282809:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762808.282811:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762808.282816:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2743 00000100:00080000:1.0:1789762808.282820:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762808.282824:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762808.282834:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762808.282846:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565810688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762808.282855:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565810560:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789762808.282858:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762808.282864:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762808.282865:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762808.282869:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2743 00000100:00080000:1.0:1789762808.282871:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762808.282873:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762808.282877:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762808.282887:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565810816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762808.282896:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762808.282899:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762808.282901:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762808.282904:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2743 00000100:00080000:1.0:1789762808.282908:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762808.282910:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762808.282913:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789762808.282917:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762808.282926:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565810944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762808.282963:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762808/real 1789762808] req@ffff93f844cca680 x1876699565810560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762808.282983:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565810560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762808.282995:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762808.283000:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762808.283004:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565810560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762808.283028:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565810688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762808.283048:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762808.283059:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762808/real 1789762808] req@ffff93f844cc9c00 x1876699565810688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762808.283073:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565810688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762808.283082:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762808.283085:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762808.283089:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565810688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762808.283097:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565810816:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762808.283117:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762808.283129:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762808/real 1789762808] req@ffff93f844ccaa00 x1876699565810816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762808.283141:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565810816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762808.283150:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762808.283154:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762808.283158:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565810816:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762808.283165:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565810944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762808.283185:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762808.283193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762808/real 1789762808] req@ffff93f844ccb480 x1876699565810944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762808.283201:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565810944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762808.283207:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762808.283209:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762808.283211:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565810944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762808.346788:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762809.370814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762810.394817:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762811.418750:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762812.442771:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762813.402735:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762813.402743:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762813.402748:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762813.402757:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2748 00000100:00080000:1.0:1789762813.402765:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762813.402771:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762813.402803:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565811072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762813.402820:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762813.402823:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762813.402824:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762813.402827:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2748 00000100:00080000:1.0:1789762813.402829:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762813.402832:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762813.402840:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565811072:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789762813.402847:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762813.402859:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565811200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762813.402866:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762813.402869:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762813.402870:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762813.402873:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2748 00000100:00080000:1.0:1789762813.402876:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762813.402878:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762813.402881:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762813.402889:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565811328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762813.402901:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762813.402902:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762813.402907:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762813.402908:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762813.402912:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2748 00000100:00080000:1.0:1789762813.402916:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762813.402921:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762813.402925:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762813.402938:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565811456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762813.402951:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762813/real 1789762813] req@ffff93f844ccb800 x1876699565811072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762813.402967:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565811072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762813.402976:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762813.402980:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762813.402983:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565811072:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762813.403003:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565811200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762813.403018:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762813.403026:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762813/real 1789762813] req@ffff93f844cc8700 x1876699565811200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762813.403034:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565811200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762813.403040:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762813.403042:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762813.403044:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565811200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762813.403048:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565811328:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762813.403058:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762813.403063:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762813/real 1789762813] req@ffff93f844ccbb80 x1876699565811328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762813.403070:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565811328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762813.403080:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762813.403082:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762813.403084:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565811328:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762813.403087:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565811456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762813.403096:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762813.403102:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762813/real 1789762813] req@ffff93f844cc8000 x1876699565811456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762813.403108:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565811456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762813.403113:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762813.403115:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762813.403117:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565811456:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762813.466797:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762814.490791:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762815.514830:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762816.538822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762817.562851:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762818.522763:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762818.522773:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762818.522777:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762818.522786:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2753 00000100:00080000:2.0:1789762818.522797:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762818.522806:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762818.522846:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565811584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762818.522884:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762818.522889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762818.522891:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762818.522895:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2753 00000100:00080000:2.0:1789762818.522899:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762818.522902:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762818.522911:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762818.522921:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565811712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762818.522928:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762818.522931:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762818.522932:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762818.522935:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2753 00000100:00080000:2.0:1789762818.522937:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762818.522940:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762818.522943:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762818.522950:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565811840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762818.522956:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762818.522959:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762818.522960:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762818.522962:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2753 00000100:00080000:2.0:1789762818.522964:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762818.522966:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762818.522972:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565811584:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789762818.522975:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762818.522985:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565811968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789762818.523032:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762818.523079:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762818/real 1789762818] req@ffff93f8517c1880 x1876699565811584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762818.523099:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565811584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762818.523109:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762818.523114:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762818.523123:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565811584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762818.523132:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565811712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762818.523157:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762818.523180:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762818/real 1789762818] req@ffff93f8517c3480 x1876699565811712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762818.523192:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565811712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762818.523198:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762818.523201:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762818.523204:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565811712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762818.523208:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565811840:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762818.523222:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762818.523234:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762818/real 1789762818] req@ffff93f8517c3800 x1876699565811840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762818.523242:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565811840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762818.523262:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762818.523266:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762818.523269:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565811840:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762818.523293:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565811968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762818.523310:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762818.523319:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762818/real 1789762818] req@ffff93f8517c0e00 x1876699565811968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762818.523327:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565811968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762818.523340:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762818.523343:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762818.523346:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565811968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762818.586749:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762819.610869:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762820.634796:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762821.658785:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762822.682803:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762823.642739:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762823.642748:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762823.642752:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762823.642761:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2758 00000100:00080000:2.0:1789762823.642768:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762823.642775:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762823.642813:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565812096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762823.642843:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762823.642847:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762823.642849:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762823.642854:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2758 00000100:00080000:2.0:1789762823.642858:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762823.642863:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762823.642874:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762823.642891:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565812224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762823.642896:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565812096:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762823.642901:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762823.642905:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762823.642907:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762823.642910:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2758 00000100:00080000:2.0:1789762823.642913:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762823.642916:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762823.642921:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762823.642930:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565812352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762823.642937:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762823.642940:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762823.642941:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762823.642943:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2758 00000100:00080000:2.0:1789762823.642946:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762823.642948:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762823.642952:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762823.642958:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565812480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789762823.642974:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762823.643032:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762823/real 1789762823] req@ffff93f8517c0e00 x1876699565812096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762823.643052:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565812096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762823.643064:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762823.643068:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762823.643071:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565812096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762823.643092:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565812224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762823.643115:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762823.643126:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762823/real 1789762823] req@ffff93f8517c3800 x1876699565812224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762823.643138:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565812224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762823.643147:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762823.643150:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762823.643154:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565812224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762823.643160:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565812352:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762823.643181:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762823.643191:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762823/real 1789762823] req@ffff93f8517c3480 x1876699565812352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762823.643203:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565812352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762823.643228:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762823.643231:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762823.643235:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565812352:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762823.643242:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565812480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762823.643262:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762823.643274:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762823/real 1789762823] req@ffff93f8517c1880 x1876699565812480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762823.643284:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565812480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762823.643293:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762823.643296:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762823.643299:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565812480:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762823.706788:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762824.730779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762825.754786:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762826.778806:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762827.802779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762828.763110:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762828.763124:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762828.763129:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762828.763141:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2763 00000100:00080000:2.0:1789762828.763155:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762828.763164:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762828.763219:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565812608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762828.763317:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762828.763322:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762828.763324:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762828.763328:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2763 00000100:00080000:2.0:1789762828.763333:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762828.763337:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762828.763352:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762828.763357:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565812608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762828.763370:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565812736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762828.763380:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762828.763382:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762828.763384:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762828.763387:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2763 00000100:00080000:2.0:1789762828.763389:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762828.763392:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762828.763395:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762828.763403:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565812864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762828.763410:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:1.0:1789762828.763415:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762828.763416:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762828.763420:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762828.763423:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2763 00000100:00080000:2.0:1789762828.763427:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762828.763430:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762828.763434:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762828.763443:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565812992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762828.763461:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762828/real 1789762828] req@ffff93f8517c2d80 x1876699565812608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762828.763476:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565812608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762828.763484:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762828.763488:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762828.763491:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565812608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762828.763510:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565812736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762828.763530:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762828.763544:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762828/real 1789762828] req@ffff93f8517c2a00 x1876699565812736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762828.763557:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565812736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762828.763567:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762828.763570:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762828.763574:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565812736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762828.763581:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565812864:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762828.763598:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762828.763607:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762828/real 1789762828] req@ffff93f8517c3100 x1876699565812864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762828.763614:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565812864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762828.763624:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762828.763627:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762828.763629:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565812864:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762828.763632:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565812992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762828.763662:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762828.763675:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762828/real 1789762828] req@ffff93f8517c1180 x1876699565812992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762828.763683:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565812992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762828.763689:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762828.763692:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762828.763694:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565812992:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762828.826859:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762829.850832:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762830.874809:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762831.898795:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762832.922799:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762833.882754:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762833.882762:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762833.882766:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762833.882773:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2768 00000100:00080000:0.0:1789762833.882781:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762833.882787:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762833.882820:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565813120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762833.882843:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762833.882846:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762833.882847:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762833.882850:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2768 00000100:00080000:0.0:1789762833.882852:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762833.882855:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762833.882862:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762833.882869:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565813248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762833.882877:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762833.882879:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762833.882880:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762833.882882:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2768 00000100:00080000:0.0:1789762833.882884:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762833.882886:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762833.882888:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762833.882894:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565813376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762833.882900:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762833.882903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762833.882904:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762833.882907:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2768 00000100:00080000:0.0:1789762833.882909:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762833.882912:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762833.882914:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762833.882920:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565813504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762833.882927:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565813120:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1789762833.882982:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762833.883023:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762833/real 1789762833] req@ffff93f850d41180 x1876699565813120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762833.883038:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565813120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762833.883046:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762833.883049:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762833.883051:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565813120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762833.883057:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565813248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762833.883071:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762833.883079:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762833/real 1789762833] req@ffff93f850d43100 x1876699565813248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762833.883086:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565813248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762833.883091:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762833.883093:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762833.883095:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565813248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762833.883099:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565813376:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762833.883110:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762833.883116:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762833/real 1789762833] req@ffff93f850d41f80 x1876699565813376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762833.883123:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565813376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762833.883128:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762833.883146:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762833.883148:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565813376:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762833.883167:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565813504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762833.883179:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762833.883187:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762833/real 1789762833] req@ffff93f850d41500 x1876699565813504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762833.883195:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565813504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762833.883200:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762833.883202:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762833.883204:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565813504:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762833.946811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762834.970890:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762835.994775:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762837.018803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762838.042765:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762839.002751:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762839.002759:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762839.002762:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762839.002770:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2773 00000100:00080000:1.0:1789762839.002778:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762839.002784:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762839.002815:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565813632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762839.002840:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762839.002844:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762839.002846:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762839.002850:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2773 00000100:00080000:1.0:1789762839.002854:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762839.002858:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762839.002867:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762839.002876:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565813632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762839.002880:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565813760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762839.002896:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762839.002898:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762839.002900:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762839.002904:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2773 00000100:00080000:1.0:1789762839.002907:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762839.002910:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762839.002914:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762839.002922:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565813888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762839.002932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1789762839.002932:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762839.002935:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762839.002936:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762839.002940:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2773 00000100:00080000:1.0:1789762839.002942:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762839.002946:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762839.002950:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762839.002958:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565814016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762839.002979:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762839/real 1789762839] req@ffff93f844cc8380 x1876699565813632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762839.002999:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565813632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762839.003009:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762839.003012:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762839.003015:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565813632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762839.003034:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565813760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762839.003061:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762839.003070:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762839/real 1789762839] req@ffff93f844cca300 x1876699565813760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762839.003078:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565813760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762839.003083:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762839.003085:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762839.003087:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565813760:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762839.003104:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565813888:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762839.003126:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762839.003143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762839/real 1789762839] req@ffff93f844cc9f80 x1876699565813888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762839.003151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565813888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762839.003156:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762839.003158:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762839.003160:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565813888:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762839.003164:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565814016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762839.003179:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762839.003185:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762839/real 1789762839] req@ffff93f844cc9880 x1876699565814016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762839.003192:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565814016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762839.003197:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762839.003199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762839.003201:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565814016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762839.066768:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762840.090839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762841.114825:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762842.138847:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762843.162804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762844.122761:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762844.122773:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762844.122778:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762844.122789:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2779 00000100:00080000:0.0:1789762844.122802:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762844.122816:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762844.122861:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565814144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762844.122886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762844.122889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762844.122891:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762844.122894:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2779 00000100:00080000:0.0:1789762844.122896:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762844.122899:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762844.122908:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762844.122919:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565814272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762844.122927:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762844.122930:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762844.122932:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762844.122935:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2779 00000100:00080000:0.0:1789762844.122937:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762844.122939:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762844.122943:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762844.122951:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565814400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762844.122960:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762844.122969:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:2.0:1789762844.122969:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565814144:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762844.122973:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762844.122992:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2779 00000100:00080000:0.0:1789762844.122997:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762844.123004:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762844.123012:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762844.123028:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565814528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762844.123071:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762844.123134:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762844/real 1789762844] req@ffff93f850d43480 x1876699565814144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762844.123161:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565814144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762844.123170:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762844.123173:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762844.123176:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565814144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762844.123183:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565814272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762844.123203:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762844.123212:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762844/real 1789762844] req@ffff93f850d43800 x1876699565814272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762844.123220:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565814272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762844.123226:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762844.123228:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762844.123230:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565814272:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762844.123249:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565814400:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762844.123261:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762844.123267:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762844/real 1789762844] req@ffff93f850d41500 x1876699565814400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762844.123275:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565814400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762844.123284:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762844.123286:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762844.123288:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565814400:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762844.123292:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565814528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762844.123303:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762844.123308:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762844/real 1789762844] req@ffff93f850d41f80 x1876699565814528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762844.123315:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565814528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762844.123320:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762844.123322:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762844.123324:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565814528:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762844.186860:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762845.210831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762846.234778:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762847.258763:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762848.282764:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762849.242720:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762849.242732:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762849.242736:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762849.242744:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2784 00000100:00080000:2.0:1789762849.242751:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762849.242757:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762849.242787:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565814656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762849.242811:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762849.242814:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762849.242816:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762849.242818:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2784 00000100:00080000:2.0:1789762849.242821:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762849.242823:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762849.242830:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762849.242839:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565814784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762849.242847:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762849.242849:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762849.242851:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762849.242854:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2784 00000100:00080000:2.0:1789762849.242856:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762849.242859:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762849.242863:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762849.242865:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565814656:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762849.242872:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565814912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762849.242880:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762849.242882:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762849.242883:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762849.242886:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2784 00000100:00080000:2.0:1789762849.242888:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762849.242890:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762849.242894:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762849.242903:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565815040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762849.242968:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565814784:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762849.243010:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565814912:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762849.243052:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565815040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762849.306754:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762850.330788:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762851.354779:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789762852.314882:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789762852.314916:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789762852.314929:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762852.314960:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762852.314966:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789762852.314972:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789762852.315023:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762849/real 1789762852] req@ffff93f8517c2680 x1876699565814912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762852.315048:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565814912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762852.315060:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762852.315065:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762852.315069:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565814912:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762852.315077:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762849/real 1789762852] req@ffff93f8517c0a80 x1876699565815040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762852.315093:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565815040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762852.315100:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762852.315103:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762852.315107:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565815040:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762852.315113:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762849/real 1789762852] req@ffff93f8517c0380 x1876699565814656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762852.315123:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565814656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762852.315130:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762852.315133:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762852.315137:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565814656:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762852.315143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762849/real 1789762852] req@ffff93f8517c1500 x1876699565814784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762852.315154:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565814784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762852.315160:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762852.315164:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762852.315167:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565814784:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762852.378789:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762853.402759:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762854.362719:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762854.362728:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762854.362732:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762854.362740:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2789 00000100:00080000:2.0:1789762854.362748:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762854.362754:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762854.362785:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565815168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762854.362812:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762854.362815:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762854.362816:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762854.362819:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2789 00000100:00080000:2.0:1789762854.362822:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762854.362825:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762854.362832:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762854.362845:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565815296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762854.362850:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565815168:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762854.362857:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762854.362860:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762854.362862:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762854.362866:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2789 00000100:00080000:2.0:1789762854.362869:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762854.362872:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762854.362877:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762854.362886:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565815424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762854.362894:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762854.362898:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762854.362901:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762854.362907:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2789 00000100:00080000:2.0:1789762854.362912:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789762854.362917:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762854.362921:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762854.362929:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762854.362943:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565815552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762854.362963:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762854/real 1789762854] req@ffff93f8517c2300 x1876699565815168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762854.362996:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565815168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762854.363007:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762854.363013:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762854.363018:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565815168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762854.363045:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565815296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762854.363068:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762854.363081:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762854/real 1789762854] req@ffff93f8517c1f80 x1876699565815296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762854.363092:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565815296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762854.363101:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762854.363105:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762854.363109:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565815296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762854.363115:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565815424:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762854.363134:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762854.363144:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762854/real 1789762854] req@ffff93f8517c3b80 x1876699565815424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762854.363157:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565815424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762854.363166:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762854.363169:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762854.363173:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565815424:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762854.363179:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565815552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762854.363198:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762854.363209:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762854/real 1789762854] req@ffff93f8517c0000 x1876699565815552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762854.363220:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565815552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762854.363228:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762854.363231:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762854.363235:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565815552:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762854.426764:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762855.450766:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762856.474814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762857.498775:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762858.522834:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762859.482776:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762859.482785:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762859.482789:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762859.482797:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2794 00000100:00080000:2.0:1789762859.482807:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762859.482816:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762859.482855:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565815680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762859.482883:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762859.482889:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762859.482892:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762859.482897:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2794 00000100:00080000:2.0:1789762859.482902:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762859.482906:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762859.482916:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762859.482924:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565815680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762859.482934:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565815808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762859.482957:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762859.482959:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762859.482960:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762859.482965:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2794 00000100:00080000:2.0:1789762859.482967:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762859.482971:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762859.482976:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762859.482990:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565815936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762859.483002:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762859.483006:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762859.483008:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762859.483013:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2794 00000800:00000100:0.0:1789762859.483019:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762859.483019:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762859.483029:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762859.483035:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762859.483049:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565816064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762859.483085:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762859/real 1789762859] req@ffff93f8517c0000 x1876699565815680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762859.483103:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565815680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762859.483113:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762859.483118:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762859.483121:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565815680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762859.483146:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565815808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762859.483163:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762859.483173:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762859/real 1789762859] req@ffff93f8517c3b80 x1876699565815808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762859.483181:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565815808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762859.483187:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762859.483189:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762859.483191:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565815808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762859.483195:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565815936:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762859.483207:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762859.483214:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762859/real 1789762859] req@ffff93f8517c1f80 x1876699565815936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762859.483221:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565815936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762859.483227:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762859.483229:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762859.483231:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565815936:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762859.483234:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565816064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762859.483245:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762859.483252:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762859/real 1789762859] req@ffff93f8517c2300 x1876699565816064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762859.483260:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565816064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762859.483265:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762859.483266:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762859.483268:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565816064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762859.546765:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762860.570815:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762861.594757:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762862.618779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762863.642874:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762864.602757:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762864.602765:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762864.602769:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762864.602777:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2799 00000100:00080000:1.0:1789762864.602785:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762864.602791:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762864.602823:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565816192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762864.602847:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762864.602849:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762864.602851:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762864.602854:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2799 00000100:00080000:1.0:1789762864.602857:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762864.602860:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762864.602867:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762864.602876:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565816192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762864.602879:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565816320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762864.602890:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762864.602892:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762864.602893:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762864.602896:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2799 00000100:00080000:1.0:1789762864.602898:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762864.602917:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762864.602923:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762864.602938:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565816448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762864.602947:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762864.602955:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762864.602960:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762864.602963:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762864.602967:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2799 00000100:00080000:1.0:1789762864.602971:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762864.602975:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762864.602979:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762864.602987:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762864/real 1789762864] req@ffff93f844cc8a80 x1876699565816192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762864.602992:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565816576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762864.603017:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565816192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762864.603027:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762864.603031:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762864.603034:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565816192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762864.603052:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565816320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762864.603067:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762864.603076:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762864/real 1789762864] req@ffff93f844cc9180 x1876699565816320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762864.603084:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565816320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762864.603089:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762864.603091:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762864.603093:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565816320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762864.603096:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565816448:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762864.603107:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762864.603113:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762864/real 1789762864] req@ffff93f844cc9880 x1876699565816448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762864.603121:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565816448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762864.603126:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762864.603127:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762864.603129:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565816448:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762864.603133:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565816576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762864.603143:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762864.603148:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762864/real 1789762864] req@ffff93f844cc9f80 x1876699565816576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762864.603155:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565816576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762864.603161:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762864.603162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762864.603164:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565816576:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762864.666794:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762865.690861:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762866.714796:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762867.738835:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762868.762801:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762869.722757:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762869.722770:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762869.722775:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762869.722789:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2804 00000100:00080000:1.0:1789762869.722801:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762869.722810:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762869.722876:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565816704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762869.722908:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762869.722912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762869.722914:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762869.722919:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2804 00000100:00080000:1.0:1789762869.722924:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762869.722929:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762869.722940:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762869.722942:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565816704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762869.722959:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565816832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762869.722972:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762869.722976:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762869.722978:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762869.722982:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2804 00000100:00080000:1.0:1789762869.722986:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762869.722990:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762869.722996:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762869.723007:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565816960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762869.723024:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1789762869.723027:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762869.723029:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762869.723032:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762869.723037:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2804 00000100:00080000:1.0:1789762869.723041:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762869.723045:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762869.723050:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762869.723061:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565817088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762869.723091:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762869/real 1789762869] req@ffff93f844cca300 x1876699565816704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762869.723113:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565816704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762869.723126:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762869.723131:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762869.723135:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565816704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762869.723163:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565816832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762869.723187:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762869.723200:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762869/real 1789762869] req@ffff93f844cc8380 x1876699565816832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762869.723214:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565816832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762869.723224:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762869.723227:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762869.723231:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565816832:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762869.723236:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565816960:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762869.723268:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762869.723280:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762869/real 1789762869] req@ffff93f844cc8000 x1876699565816960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762869.723294:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565816960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762869.723303:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762869.723307:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762869.723311:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565816960:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762869.723316:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565817088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762869.723334:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762869.723345:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762869/real 1789762869] req@ffff93f844ccbb80 x1876699565817088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762869.723363:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565817088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762869.723375:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762869.723379:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762869.723383:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565817088:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762869.786855:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762870.810796:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762871.834910:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762872.858784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762873.882857:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762874.842845:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762874.842871:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762874.842875:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762874.842883:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2809 00000100:00080000:2.0:1789762874.842892:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762874.842898:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762874.842932:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565817216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762874.842959:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762874.842962:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762874.842963:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762874.842966:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2809 00000100:00080000:2.0:1789762874.842970:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762874.842973:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762874.842981:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762874.842993:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565817344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762874.843006:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565817216:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762874.843006:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762874.843020:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762874.843023:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762874.843027:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2809 00000100:00080000:2.0:1789762874.843030:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762874.843032:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762874.843036:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762874.843047:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565817472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762874.843055:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762874.843057:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762874.843059:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762874.843062:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2809 00000100:00080000:2.0:1789762874.843065:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762874.843067:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762874.843071:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762874.843077:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565817600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762874.843098:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762874.843148:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762874/real 1789762874] req@ffff93f8517c1500 x1876699565817216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762874.843163:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565817216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762874.843173:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762874.843177:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762874.843180:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565817216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762874.843205:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565817344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762874.843222:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762874.843230:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762874/real 1789762874] req@ffff93f8517c0380 x1876699565817344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762874.843238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565817344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762874.843244:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762874.843246:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762874.843248:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565817344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762874.843254:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565817472:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762874.843266:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762874.843273:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762874/real 1789762874] req@ffff93f8517c0a80 x1876699565817472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762874.843280:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565817472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762874.843286:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762874.843288:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762874.843290:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565817472:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762874.843293:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565817600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762874.843304:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762874.843311:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762874/real 1789762874] req@ffff93f8517c2680 x1876699565817600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762874.843318:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565817600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762874.843324:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762874.843327:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762874.843330:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565817600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762874.906808:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762875.930841:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762876.954909:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762877.978761:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762879.002806:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762879.962757:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762879.962766:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762879.962771:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762879.962783:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2814 00000100:00080000:2.0:1789762879.962806:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762879.962815:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762879.962861:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565817728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762879.962886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762879.962892:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762879.962893:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762879.962896:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2814 00000100:00080000:2.0:1789762879.962900:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762879.962903:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762879.962913:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762879.962924:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565817856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762879.962931:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565817728:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762879.962942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762879.962946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762879.962947:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762879.962950:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2814 00000100:00080000:2.0:1789762879.962953:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762879.962955:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762879.962959:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762879.962968:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565817984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762879.962976:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762879.962979:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762879.962981:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762879.962987:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2814 00000100:00080000:2.0:1789762879.962996:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762879.963000:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762879.963004:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789762879.963010:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762879.963019:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565818112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762879.963060:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762879/real 1789762879] req@ffff93f8517c1180 x1876699565817728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762879.963076:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565817728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762879.963085:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762879.963089:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762879.963092:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565817728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762879.963111:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565817856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762879.963127:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762879.963136:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762879/real 1789762879] req@ffff93f8517c3100 x1876699565817856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762879.963144:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565817856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762879.963150:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762879.963152:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762879.963153:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565817856:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762879.963157:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565817984:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762879.963169:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762879.963175:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762879/real 1789762879] req@ffff93f8517c2a00 x1876699565817984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762879.963183:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565817984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762879.963192:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762879.963194:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762879.963196:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565817984:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762879.963200:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565818112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762879.963210:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762879.963216:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762879/real 1789762879] req@ffff93f8517c2d80 x1876699565818112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762879.963223:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565818112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762879.963228:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762879.963230:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762879.963232:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565818112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762880.026838:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762881.050748:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762882.074787:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762883.098854:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762884.122787:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762885.082800:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762885.082816:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762885.082821:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762885.082833:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2820 00000100:00080000:1.0:1789762885.082846:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762885.082855:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762885.082902:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565818240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762885.082932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762885.082937:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762885.082939:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762885.082945:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2820 00000100:00080000:1.0:1789762885.082950:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762885.082955:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762885.082969:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762885.082982:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565818240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762885.082988:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565818368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762885.083009:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762885.083012:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762885.083015:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762885.083023:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2820 00000100:00080000:1.0:1789762885.083027:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762885.083032:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762885.083038:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762885.083052:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565818496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762885.083065:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762885.083070:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762885.083073:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789762885.083081:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762885.083089:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2820 00000100:00080000:1.0:1789762885.083096:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762885.083102:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762885.083109:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762885.083126:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565818624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762885.083131:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762885/real 1789762885] req@ffff93f844cc8700 x1876699565818240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762885.083191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565818240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762885.083202:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762885.083206:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762885.083219:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565818240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762885.083242:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565818368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762885.083259:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762885.083268:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762885/real 1789762885] req@ffff93f844ccb800 x1876699565818368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762885.083277:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565818368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762885.083283:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762885.083285:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762885.083288:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565818368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762885.083292:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565818496:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762885.083304:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762885.083310:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762885/real 1789762885] req@ffff93f844ccb480 x1876699565818496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762885.083324:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565818496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762885.083330:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762885.083333:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762885.083335:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565818496:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762885.083339:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565818624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762885.083350:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762885.083357:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762885/real 1789762885] req@ffff93f844ccaa00 x1876699565818624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762885.083365:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565818624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762885.083371:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762885.083373:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762885.083377:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565818624:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762885.146805:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762886.170770:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762887.194867:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762888.218833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762889.242895:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762890.202779:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762890.202791:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762890.202797:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762890.202809:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2825 00000100:00080000:1.0:1789762890.202821:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762890.202831:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762890.202876:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565818752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762890.202906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762890.202910:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762890.202912:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762890.202916:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2825 00000100:00080000:1.0:1789762890.202920:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762890.202923:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762890.202935:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762890.202945:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565818752:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762890.202950:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565818880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762890.202961:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762890.202964:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762890.202965:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762890.202968:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2825 00000100:00080000:1.0:1789762890.202972:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762890.202975:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762890.202979:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762890.202992:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565819008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762890.203013:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762890.203016:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762890.203019:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762890.203024:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2825 00000100:00080000:1.0:1789762890.203028:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762890.203033:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762890.203039:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789762890.203052:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762890.203053:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565819136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762890.203211:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762890/real 1789762890] req@ffff93f844cc9c00 x1876699565818752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762890.203391:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565818752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762890.203405:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762890.203411:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762890.203416:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565818752:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762890.203487:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565818880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762890.203510:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762890.203521:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762890/real 1789762890] req@ffff93f844cca680 x1876699565818880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762890.203531:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565818880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762890.203544:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762890.203547:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762890.203550:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565818880:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762890.203554:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565819008:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762890.203568:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762890.203576:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762890/real 1789762890] req@ffff93f844cc8e00 x1876699565819008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762890.203584:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565819008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762890.203594:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762890.203597:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762890.203599:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565819008:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762890.203603:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565819136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762890.203614:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762890.203620:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762890/real 1789762890] req@ffff93f844ccb100 x1876699565819136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762890.203627:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565819136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762890.203633:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762890.203635:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762890.203636:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565819136:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762890.266822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762891.290842:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762892.314901:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762893.338882:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762894.362844:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762895.322786:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762895.322804:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762895.322811:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762895.322826:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2830 00000100:00080000:2.0:1789762895.322839:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762895.322851:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762895.322911:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565819264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762895.322948:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762895.322952:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762895.322953:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762895.322958:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2830 00000100:00080000:2.0:1789762895.322962:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762895.322967:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762895.322978:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762895.322995:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565819392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762895.323011:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1789762895.323014:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565819264:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762895.323016:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762895.323019:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762895.323030:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2830 00000100:00080000:2.0:1789762895.323037:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762895.323041:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762895.323047:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762895.323066:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565819520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762895.323080:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762895.323084:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762895.323086:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762895.323091:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2830 00000100:00080000:2.0:1789762895.323104:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762895.323110:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762895.323117:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789762895.323126:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762895.323137:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565819648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762895.323218:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762895/real 1789762895] req@ffff93f8517c1880 x1876699565819264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762895.323248:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565819264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762895.323266:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762895.323273:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762895.323280:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565819264:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762895.323329:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565819392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762895.323389:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762895.323410:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762895/real 1789762895] req@ffff93f8517c3480 x1876699565819392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762895.323427:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565819392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762895.323440:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762895.323444:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762895.323449:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565819392:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762895.323458:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565819520:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789762895.323488:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762895.323504:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762895/real 1789762895] req@ffff93f8517c3800 x1876699565819520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762895.323523:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565819520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762895.323535:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762895.323540:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762895.323545:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565819520:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762895.323554:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565819648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789762895.323585:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762895.323602:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762895/real 1789762895] req@ffff93f8517c0e00 x1876699565819648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762895.323621:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565819648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762895.323634:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762895.323639:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762895.323702:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565819648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762895.386869:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762896.410871:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762897.434860:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762898.458826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762899.482809:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762900.442780:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762900.442788:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762900.442792:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762900.442800:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2835 00000100:00080000:2.0:1789762900.442810:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762900.442817:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762900.442849:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565819776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762900.442874:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762900.442879:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762900.442880:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762900.442883:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2835 00000100:00080000:2.0:1789762900.442886:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762900.442890:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762900.442899:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762900.442907:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565819904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762900.442919:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789762900.442921:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565819776:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762900.442923:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762900.442925:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762900.442944:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2835 00000100:00080000:2.0:1789762900.442949:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762900.442955:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762900.442962:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762900.442980:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565820032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762900.442993:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762900.442996:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762900.442998:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762900.443002:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2835 00000100:00080000:2.0:1789762900.443006:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762900.443009:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762900.443016:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789762900.443017:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762900.443036:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565820160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762900.443072:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762900/real 1789762900] req@ffff93f8517c0e00 x1876699565819776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762900.443089:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565819776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762900.443098:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762900.443102:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762900.443106:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565819776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762900.443130:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565819904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762900.443146:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762900.443155:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762900/real 1789762900] req@ffff93f8517c3800 x1876699565819904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762900.443163:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565819904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762900.443174:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762900.443176:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762900.443179:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565819904:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762900.443183:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565820032:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762900.443194:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762900.443200:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762900/real 1789762900] req@ffff93f8517c3480 x1876699565820032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762900.443207:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565820032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762900.443213:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762900.443216:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762900.443218:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565820032:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762900.443221:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565820160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762900.443232:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762900.443238:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762900/real 1789762900] req@ffff93f8517c1880 x1876699565820160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762900.443244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565820160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762900.443250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762900.443252:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762900.443254:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565820160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762900.506790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762901.530815:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762902.554787:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762903.578790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762904.602849:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762905.562774:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762905.562783:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762905.562787:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762905.562797:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2840 00000100:00080000:1.0:1789762905.562807:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762905.562816:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762905.562853:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565820288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762905.562879:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762905.562883:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762905.562885:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762905.562889:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2840 00000100:00080000:1.0:1789762905.562900:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762905.562904:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762905.562916:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762905.562924:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565820288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762905.562933:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565820416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762905.562946:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762905.562949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762905.562951:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762905.562955:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2840 00000100:00080000:1.0:1789762905.562958:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762905.562961:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762905.562966:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762905.562978:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565820544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762905.562988:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1789762905.562996:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789762905.562997:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762905.563004:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762905.563009:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2840 00000100:00080000:1.0:1789762905.563013:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762905.563017:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762905.563022:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762905.563034:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565820672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762905.563054:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762905/real 1789762905] req@ffff93f844cc9500 x1876699565820288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762905.563073:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565820288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762905.563084:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762905.563088:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762905.563092:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565820288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762905.563113:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565820416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762905.563134:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762905.563145:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762905/real 1789762905] req@ffff93f844ccad80 x1876699565820416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762905.563156:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565820416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762905.563163:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762905.563166:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762905.563169:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565820416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762905.563174:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565820544:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762905.563189:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762905.563198:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762905/real 1789762905] req@ffff93f844ccb100 x1876699565820544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762905.563209:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565820544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762905.563216:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762905.563218:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762905.563221:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565820544:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762905.563226:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565820672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762905.563240:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762905.563248:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762905/real 1789762905] req@ffff93f844cc8e00 x1876699565820672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762905.563258:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565820672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762905.563264:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762905.563267:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762905.563269:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565820672:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762905.626801:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762906.650821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762907.674773:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762908.698772:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762909.722833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789762910.682747:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762910.682759:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789762910.682766:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762910.682779:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2845 00000100:00080000:1.0:1789762910.682792:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762910.682813:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762910.682872:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565820800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762910.682906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762910.682909:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762910.682911:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762910.682914:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2845 00000100:00080000:1.0:1789762910.682917:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762910.682921:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762910.682931:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762910.682940:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565820800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762910.682945:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565820928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762910.682966:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762910.682968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789762910.682969:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762910.682971:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2845 00000100:00080000:1.0:1789762910.682973:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762910.682975:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762910.682979:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789762910.682987:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565821056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789762910.683001:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789762910.683004:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789762910.683005:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789762910.683008:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2845 00000100:00080000:1.0:1789762910.683009:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789762910.683012:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789762910.683015:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762910.683031:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565821184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762910.683047:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762910.683100:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762910/real 1789762910] req@ffff93f844cca680 x1876699565820800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762910.683117:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565820800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762910.683127:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762910.683130:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762910.683140:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565820800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762910.683167:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565820928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762910.683189:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762910.683199:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762910/real 1789762910] req@ffff93f844cc9c00 x1876699565820928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762910.683208:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565820928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762910.683215:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762910.683217:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762910.683219:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565820928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762910.683224:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565821056:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762910.683238:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762910.683244:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762910/real 1789762910] req@ffff93f844ccaa00 x1876699565821056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762910.683252:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565821056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762910.683258:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762910.683260:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762910.683262:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565821056:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762910.683266:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565821184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762910.683278:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762910.683284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762910/real 1789762910] req@ffff93f844ccb480 x1876699565821184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762910.683292:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565821184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762910.683297:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762910.683299:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762910.683301:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565821184:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762910.746929:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762911.770823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762912.794769:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762913.818805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762914.842841:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762915.802727:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762915.802737:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762915.802742:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762915.802753:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2850 00000100:00080000:2.0:1789762915.802764:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762915.802772:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762915.802814:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565821312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762915.802839:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762915.802842:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762915.802844:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762915.802847:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2850 00000100:00080000:2.0:1789762915.802850:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762915.802853:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762915.802862:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762915.802873:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565821440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762915.802881:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565821312:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762915.802884:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762915.802890:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762915.802892:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762915.802896:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2850 00000100:00080000:2.0:1789762915.802899:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762915.802902:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762915.802906:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762915.802915:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565821568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762915.802927:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762915.802931:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762915.802933:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762915.802937:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2850 00000100:00080000:2.0:1789762915.802942:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762915.802947:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762915.802953:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762915.802967:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565821696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762915.803007:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565821440:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762915.803040:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565821568:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762915.803061:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565821696:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762915.866778:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762916.890814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762917.914784:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789762918.874980:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789762918.875024:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789762918.875039:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789762918.875094:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789762918.875108:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789762918.875123:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1789762918.875152:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762915/real 1789762918] req@ffff93f8517c2a00 x1876699565821440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762918.875188:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565821440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762918.875207:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762918.875214:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762918.875220:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565821440:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762918.875230:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762915/real 1789762918] req@ffff93f8517c3100 x1876699565821568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762918.875247:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565821568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762918.875259:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762918.875264:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762918.875267:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565821568:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789762918.875273:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762915/real 1789762918] req@ffff93f8517c1180 x1876699565821696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762918.875286:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565821696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762918.875297:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762918.875301:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762918.875306:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565821696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789762918.875314:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762915/real 1789762918] req@ffff93f8517c2d80 x1876699565821312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762918.875330:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565821312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762918.875340:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789762918.875343:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789762918.875346:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565821312:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789762918.938801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762919.962743:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762920.922753:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762920.922762:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762920.922765:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762920.922775:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2855 00000100:00080000:3.0:1789762920.922785:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762920.922791:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762920.922822:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565821824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762920.922841:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762920.922844:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762920.922846:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762920.922848:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2855 00000100:00080000:3.0:1789762920.922851:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762920.922853:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762920.922861:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762920.922869:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565821952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762920.922877:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789762920.922880:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565821824:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762920.922880:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762920.922883:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762920.922890:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2855 00000100:00080000:3.0:1789762920.922895:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762920.922898:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762920.922901:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762920.922908:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565822080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762920.922914:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762920.922916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762920.922917:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762920.922920:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2855 00000100:00080000:3.0:1789762920.922923:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762920.922925:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762920.922928:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762920.922935:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565822208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762920.922956:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762920.922999:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762920/real 1789762920] req@ffff93f844d67480 x1876699565821824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762920.923018:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565821824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762920.923029:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762920.923034:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762920.923038:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565821824:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762920.923062:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565821952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762920.923087:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762920.923099:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762920/real 1789762920] req@ffff93f844d67800 x1876699565821952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762920.923113:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565821952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762920.923129:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762920.923132:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762920.923140:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565821952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762920.923146:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565822080:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762920.923168:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762920.923180:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762920/real 1789762920] req@ffff93f844d67100 x1876699565822080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762920.923198:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565822080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762920.923207:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762920.923210:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762920.923213:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565822080:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762920.923219:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565822208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762920.923238:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762920.923249:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762920/real 1789762920] req@ffff93f844d66a00 x1876699565822208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762920.923261:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565822208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762920.923269:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762920.923272:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762920.923274:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565822208:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762920.986816:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762922.010851:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762923.034796:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762924.058828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762925.082793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762926.042728:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762926.042739:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762926.042744:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762926.042755:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2861 00000100:00080000:3.0:1789762926.042766:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762926.042775:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762926.042828:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565822336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762926.042857:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762926.042860:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762926.042861:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762926.042865:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2861 00000100:00080000:3.0:1789762926.042867:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762926.042871:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762926.042882:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762926.042900:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565822464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762926.042911:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565822336:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762926.042913:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762926.042930:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762926.042933:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762926.042938:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2861 00000100:00080000:3.0:1789762926.042941:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762926.042945:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762926.042951:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762926.042966:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565822592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762926.042975:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762926.042979:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762926.042980:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762926.042986:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2861 00000100:00080000:3.0:1789762926.042995:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762926.042999:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762926.043003:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762926.043013:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565822720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762926.043017:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762926.043072:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762926/real 1789762926] req@ffff93f844d65c00 x1876699565822336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762926.043096:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565822336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762926.043111:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762926.043120:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762926.043129:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565822336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762926.043159:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565822464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762926.043186:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762926.043200:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762926/real 1789762926] req@ffff93f844d64e00 x1876699565822464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762926.043213:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565822464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762926.043223:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762926.043228:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762926.043231:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565822464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762926.043242:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565822592:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762926.043274:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762926.043294:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762926/real 1789762926] req@ffff93f844d66d80 x1876699565822592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762926.043310:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565822592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762926.043317:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762926.043321:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762926.043325:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565822592:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762926.043334:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565822720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762926.043363:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762926.043384:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762926/real 1789762926] req@ffff93f844d65500 x1876699565822720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762926.043396:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565822720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762926.043404:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762926.043407:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762926.043411:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565822720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762926.106821:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762927.130777:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762928.154878:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762929.178860:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762930.202875:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762931.162764:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762931.162775:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762931.162781:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762931.162793:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2866 00000100:00080000:3.0:1789762931.162803:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762931.162812:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762931.162855:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565822848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762931.162881:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762931.162885:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762931.162887:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762931.162894:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2866 00000100:00080000:3.0:1789762931.162899:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762931.162905:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762931.162919:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762931.162923:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565822848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762931.162940:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565822976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762931.162957:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762931.162962:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762931.162965:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762931.162971:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2866 00000100:00080000:3.0:1789762931.162984:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762931.162990:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762931.162997:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789762931.162998:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762931.163018:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565823104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762931.163043:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762931.163046:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762931.163048:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762931.163054:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2866 00000100:00080000:3.0:1789762931.163060:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789762931.163066:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762931/real 1789762931] req@ffff93f844d64380 x1876699565822848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762931.163071:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762931.163085:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762931.163106:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565822848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789762931.163108:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565823232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762931.163156:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762931.163163:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762931.163169:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565822848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762931.163198:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565822976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762931.163225:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762931.163239:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762931/real 1789762931] req@ffff93f844d64000 x1876699565822976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762931.163254:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565822976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762931.163267:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762931.163269:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762931.163272:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565822976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762931.163277:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565823104:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762931.163294:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762931.163302:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762931/real 1789762931] req@ffff93f844d64a80 x1876699565823104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762931.163309:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565823104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762931.163315:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762931.163317:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762931.163319:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565823104:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762931.163323:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565823232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762931.163334:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762931.163339:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762931/real 1789762931] req@ffff93f844d65f80 x1876699565823232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762931.163346:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565823232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762931.163351:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762931.163353:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762931.163355:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565823232:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762931.226782:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762932.250935:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762933.274778:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762934.298881:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762935.322862:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762936.282781:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762936.282798:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762936.282805:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762936.282821:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2871 00000100:00080000:3.0:1789762936.282837:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762936.282849:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762936.282918:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565823360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762936.282975:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762936.282991:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762936.282995:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762936.283002:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2871 00000100:00080000:3.0:1789762936.283008:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762936.283016:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762936.283032:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762936.283056:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565823488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762936.283064:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565823360:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789762936.283090:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762936.283096:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762936.283099:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762936.283106:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2871 00000100:00080000:3.0:1789762936.283112:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762936.283117:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762936.283125:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762936.283145:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565823616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762936.283160:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762936.283165:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762936.283168:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762936.283173:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2871 00000100:00080000:3.0:1789762936.283177:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762936.283182:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1789762936.283192:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1789762936.283196:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762936.283217:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565823744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762936.283264:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762936/real 1789762936] req@ffff93f844d66300 x1876699565823360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762936.283291:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565823360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762936.283306:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762936.283311:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762936.283317:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565823360:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762936.283352:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565823488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762936.283419:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762936.283439:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762936/real 1789762936] req@ffff93f844d65180 x1876699565823488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762936.283457:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565823488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762936.283467:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762936.283472:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762936.283476:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565823488:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762936.283495:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565823616:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762936.283522:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762936.283537:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762936/real 1789762936] req@ffff93f844d65f80 x1876699565823616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762936.283552:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565823616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762936.283562:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762936.283565:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762936.283569:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565823616:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762936.283576:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565823744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762936.283596:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762936.283607:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762936/real 1789762936] req@ffff93f844d64a80 x1876699565823744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762936.283621:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565823744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762936.283631:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762936.283634:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762936.283637:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565823744:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762936.346902:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762937.370777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762938.394875:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762939.418757:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762940.442791:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762941.402845:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762941.402862:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762941.402869:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762941.402884:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2876 00000100:00080000:0.0:1789762941.402898:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762941.402907:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762941.402959:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565823872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762941.403000:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762941.403005:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762941.403008:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762941.403013:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2876 00000100:00080000:0.0:1789762941.403017:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762941.403021:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762941.403032:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762941.403050:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565824000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762941.403064:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762941.403068:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762941.403069:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762941.403072:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2876 00000100:00080000:0.0:1789762941.403077:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762941.403082:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762941.403086:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762941.403096:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565823872:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762941.403097:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565824128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762941.403135:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762941.403138:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762941.403139:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762941.403142:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2876 00000100:00080000:0.0:1789762941.403146:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762941.403149:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762941.403154:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762941.403165:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565824256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789762941.403242:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762941.403374:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762941/real 1789762941] req@ffff93f850d43100 x1876699565823872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762941.403415:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565823872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762941.403439:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762941.403448:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762941.403457:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565823872:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762941.403500:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565824000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762941.403548:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762941.403578:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762941/real 1789762941] req@ffff93f850d41180 x1876699565824000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762941.403688:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565824000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762941.403710:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762941.403719:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762941.403728:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565824000:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762941.403745:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565824128:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762941.403797:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762941.403824:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762941/real 1789762941] req@ffff93f850d41c00 x1876699565824128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762941.403849:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565824128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762941.403866:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762941.403872:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762941.403880:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565824128:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762941.403894:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565824256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762941.403936:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762941.403959:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762941/real 1789762941] req@ffff93f850d42680 x1876699565824256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762941.403983:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565824256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762941.403996:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762941.404002:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762941.404008:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565824256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762941.466835:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762942.490821:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762943.514848:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762944.538779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762945.562830:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762946.522831:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762946.522844:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762946.522849:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762946.522862:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2881 00000100:00080000:2.0:1789762946.522873:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762946.522883:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762946.522930:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565824384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762946.522984:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762946.522990:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762946.522993:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762946.522998:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2881 00000100:00080000:2.0:1789762946.523002:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762946.523008:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762946.523020:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762946.523037:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565824512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762946.523050:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762946.523055:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762946.523057:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762946.523062:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2881 00000100:00080000:2.0:1789762946.523067:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762946.523072:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762946.523081:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565824384:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789762946.523083:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762946.523117:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565824640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762946.523133:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762946.523138:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762946.523141:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762946.523147:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2881 00000100:00080000:2.0:1789762946.523152:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762946.523155:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762946.523161:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762946.523177:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565824768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789762946.523191:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762946.523262:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762946/real 1789762946] req@ffff93f8517c2680 x1876699565824384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762946.523288:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565824384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762946.523324:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762946.523330:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762946.523336:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565824384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762946.523346:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565824512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762946.523382:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762946.523401:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762946/real 1789762946] req@ffff93f8517c0a80 x1876699565824512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762946.523417:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565824512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762946.523428:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762946.523433:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762946.523440:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565824512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789762946.523482:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565824640:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789762946.523526:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762946.523551:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762946/real 1789762946] req@ffff93f8517c0380 x1876699565824640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762946.523576:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565824640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762946.523594:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762946.523601:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762946.523624:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565824640:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789762946.523637:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565824768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789762946.523714:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762946.523741:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762946/real 1789762946] req@ffff93f8517c1500 x1876699565824768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762946.523766:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565824768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789762946.523783:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789762946.523790:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789762946.523797:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565824768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762946.586817:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762947.610904:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762948.634866:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762949.658911:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762950.682840:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762951.642750:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762951.642758:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762951.642762:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762951.642772:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2886 00000100:00080000:0.0:1789762951.642782:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762951.642788:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762951.642817:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565824896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762951.642840:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762951.642843:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762951.642844:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762951.642847:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2886 00000100:00080000:0.0:1789762951.642850:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762951.642852:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762951.642861:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762951.642869:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565825024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762951.642877:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565824896:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762951.642879:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762951.642886:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762951.642888:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762951.642892:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2886 00000100:00080000:0.0:1789762951.642895:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762951.642899:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762951.642905:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762951.642919:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565825152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762951.642937:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1789762951.642938:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762951.642944:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762951.642947:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762951.642953:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2886 00000100:00080000:0.0:1789762951.642959:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762951.642962:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762951.642967:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762951.642979:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762951/real 1789762951] req@ffff93f850d40e00 x1876699565824896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789762951.642981:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565825280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762951.643007:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565824896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762951.643017:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762951.643021:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762951.643025:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565824896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762951.643043:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565825024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762951.643058:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762951.643066:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762951/real 1789762951] req@ffff93f850d40000 x1876699565825024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762951.643074:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565825024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762951.643080:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762951.643082:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762951.643084:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565825024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762951.643087:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565825152:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762951.643098:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762951.643103:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762951/real 1789762951] req@ffff93f850d42300 x1876699565825152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762951.643115:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565825152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762951.643120:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762951.643122:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762951.643124:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565825152:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762951.643127:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565825280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762951.643137:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762951.643143:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762951/real 1789762951] req@ffff93f850d40380 x1876699565825280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762951.643151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565825280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762951.643159:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762951.643162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762951.643166:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565825280:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762951.706770:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762952.730910:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762953.754828:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762954.778803:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762955.802830:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762956.762857:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762956.762878:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762956.762887:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762956.762907:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2891 00000100:00080000:0.0:1789762956.762925:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762956.762941:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762956.763013:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565825408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762956.763070:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762956.763078:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762956.763082:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762956.763098:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2891 00000100:00080000:0.0:1789762956.763105:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762956.763113:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762956.763132:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762956.763160:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565825536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762956.763177:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565825408:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762956.763205:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762956.763213:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762956.763215:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762956.763244:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2891 00000100:00080000:0.0:1789762956.763250:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762956.763273:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762956.763285:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762956.763312:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565825664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762956.763333:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1789762956.763338:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789762956.763344:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762956.763350:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762956.763360:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2891 00000100:00080000:0.0:1789762956.763369:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762956.763377:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762956.763387:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762956.763409:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565825792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762956.763430:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762956/real 1789762956] req@ffff93f850d40700 x1876699565825408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762956.763460:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565825408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762956.763484:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762956.763489:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762956.763494:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565825408:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762956.763536:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565825536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762956.763578:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762956.763602:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762956/real 1789762956] req@ffff93f850d43b80 x1876699565825536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762956.763619:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565825536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762956.763629:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762956.763634:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762956.763654:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565825536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762956.763660:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565825664:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789762956.763724:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762956.763741:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762956/real 1789762956] req@ffff93f850d40380 x1876699565825664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762956.763765:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565825664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762956.763778:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762956.763783:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762956.763787:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565825664:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762956.763796:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565825792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789762956.763843:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789762956.763858:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762956/real 1789762956] req@ffff93f850d42300 x1876699565825792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762956.763871:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565825792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762956.763880:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762956.763884:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762956.763888:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565825792:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762956.826845:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762957.850783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762958.874800:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762959.898794:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762960.922774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762961.882822:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762961.882843:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762961.882849:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762961.882864:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2896 00000100:00080000:3.0:1789762961.882879:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762961.882889:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762961.882944:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565825920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762961.882975:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762961.882980:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762961.882982:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762961.882988:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2896 00000100:00080000:3.0:1789762961.882992:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762961.882997:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762961.883010:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762961.883014:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565825920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762961.883029:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565826048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762961.883041:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762961.883046:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762961.883047:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762961.883052:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2896 00000100:00080000:3.0:1789762961.883057:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762961.883061:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762961.883068:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762961.883081:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565826176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762961.883092:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762961.883097:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762961.883099:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1789762961.883118:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789762961.883162:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2896 00000100:00080000:3.0:1789762961.883168:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1789762961.883171:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762961/real 1789762961] req@ffff93f844d64000 x1876699565825920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789762961.883176:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762961.883201:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762961.883211:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565825920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762961.883222:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762961.883226:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762961.883230:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565825920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762961.883255:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565826048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762961.883278:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762961.883288:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762961/real 1789762961] req@ffff93f844d64380 x1876699565826048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762961.883299:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565826048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762961.883305:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762961.883308:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762961.883310:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565826048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762961.883315:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565826176:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762961.883333:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789762961.883339:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565826304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762961.883344:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762961/real 1789762961] req@ffff93f844d65500 x1876699565826176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762961.883371:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565826176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762961.883380:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762961.883384:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762961.883389:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565826176:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762961.883411:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565826304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762961.883437:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762961.883451:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762961/real 1789762961] req@ffff93f844d66d80 x1876699565826304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762961.883462:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565826304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762961.883473:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762961.883475:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762961.883478:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565826304:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762961.946797:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762962.970888:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762963.994765:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762965.018880:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762966.042908:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762967.002804:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762967.002817:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762967.002823:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762967.002835:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2901 00000100:00080000:2.0:1789762967.002847:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762967.002857:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762967.002904:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565826432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762967.002943:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762967.002948:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762967.002951:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762967.002955:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2901 00000100:00080000:2.0:1789762967.002959:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762967.002965:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762967.002977:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762967.002993:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565826560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762967.003009:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762967.003013:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762967.003016:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789762967.003026:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565826432:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762967.003026:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2901 00000100:00080000:2.0:1789762967.003036:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762967.003043:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762967.003050:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762967.003066:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565826688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762967.003079:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762967.003083:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762967.003086:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762967.003091:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2901 00000100:00080000:2.0:1789762967.003095:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762967.003099:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762967.003106:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762967.003122:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565826816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789762967.003128:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762967.003232:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762967/real 1789762967] req@ffff93f8517c2300 x1876699565826432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762967.003262:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565826432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762967.003278:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762967.003283:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762967.003289:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565826432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762967.003318:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565826560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762967.003347:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762967.003363:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762967/real 1789762967] req@ffff93f8517c1f80 x1876699565826560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762967.003378:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565826560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762967.003388:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762967.003393:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762967.003397:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565826560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762967.003405:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565826688:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762967.003426:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762967.003437:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762967/real 1789762967] req@ffff93f8517c3b80 x1876699565826688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762967.003451:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565826688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762967.003461:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762967.003464:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762967.003468:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565826688:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762967.003474:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565826816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762967.003493:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762967.003503:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762967/real 1789762967] req@ffff93f8517c0000 x1876699565826816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762967.003517:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565826816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762967.003527:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762967.003530:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762967.003534:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565826816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762967.066857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762968.090819:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762969.114775:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762970.138898:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762971.162879:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762972.122792:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762972.122805:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762972.122810:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762972.122822:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2907 00000100:00080000:3.0:1789762972.122832:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762972.122840:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762972.122884:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565826944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762972.122916:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762972.122919:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762972.122920:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762972.122924:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2907 00000100:00080000:3.0:1789762972.122926:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762972.122930:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762972.122940:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762972.122950:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565826944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789762972.122951:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565827072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762972.122964:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762972.122966:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762972.122967:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762972.122970:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2907 00000100:00080000:3.0:1789762972.122972:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762972.122974:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762972.122977:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762972.122986:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565827200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762972.122993:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762972.122995:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762972.122997:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762972.123000:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2907 00000100:00080000:3.0:1789762972.123004:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762972.123007:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762972.123010:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762972.123017:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565827328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789762972.123029:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762972.123086:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762972/real 1789762972] req@ffff93f844d64e00 x1876699565826944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762972.123125:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565826944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789762997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762972.123139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762972.123144:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762972.123148:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565826944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762972.123180:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565827072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762972.123202:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762972.123213:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762972/real 1789762972] req@ffff93f844d65c00 x1876699565827072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762972.123224:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565827072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762972.123233:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762972.123237:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762972.123240:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565827072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762972.123247:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565827200:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762972.123270:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762972.123280:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762972/real 1789762972] req@ffff93f844d66a00 x1876699565827200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762972.123289:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565827200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789762997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762972.123299:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762972.123302:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762972.123304:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565827200:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762972.123307:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565827328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762972.123321:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762972.123327:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762972/real 1789762972] req@ffff93f844d67100 x1876699565827328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762972.123334:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565827328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789762997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762972.123340:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762972.123341:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762972.123343:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565827328:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762972.186813:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762973.210837:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762974.234817:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762975.258864:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762976.282869:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762977.242825:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762977.242835:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762977.242840:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762977.242851:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2912 00000100:00080000:0.0:1789762977.242863:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762977.242871:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762977.242917:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565827456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762977.242948:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762977.242952:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762977.242954:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762977.242959:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2912 00000100:00080000:0.0:1789762977.242963:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762977.242967:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762977.242981:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762977.242997:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565827584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762977.243013:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565827456:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789762977.243034:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762977.243040:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762977.243042:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762977.243050:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2912 00000100:00080000:0.0:1789762977.243057:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762977.243062:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762977.243083:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762977.243098:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565827712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762977.243124:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762977.243126:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762977.243128:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762977.243131:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2912 00000100:00080000:0.0:1789762977.243142:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762977.243146:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762977.243150:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789762977.243159:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789762977.243164:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565827840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762977.243247:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762977/real 1789762977] req@ffff93f850d40000 x1876699565827456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762977.243279:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565827456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762977.243295:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762977.243302:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762977.243308:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565827456:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762977.243342:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565827584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762977.243382:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762977.243403:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762977/real 1789762977] req@ffff93f850d40e00 x1876699565827584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762977.243423:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565827584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762977.243452:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762977.243456:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762977.243461:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565827584:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762977.243471:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565827712:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762977.243514:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762977.243530:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762977/real 1789762977] req@ffff93f850d42680 x1876699565827712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762977.243553:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565827712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762977.243566:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762977.243571:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762977.243575:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565827712:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762977.243585:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565827840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762977.243616:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762977.243634:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762977/real 1789762977] req@ffff93f850d41c00 x1876699565827840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762977.243684:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565827840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762977.243696:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762977.243700:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762977.243705:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565827840:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762977.306802:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762978.330857:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762979.354802:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762980.378885:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762981.402880:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789762982.362848:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762982.362869:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789762982.362876:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762982.362892:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2917 00000100:00080000:3.0:1789762982.362908:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762982.362920:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789762982.362981:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565827968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762982.363026:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762982.363031:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762982.363060:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762982.363071:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2917 00000100:00080000:3.0:1789762982.363078:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762982.363095:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762982.363109:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565827968:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789762982.363112:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762982.363142:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565828096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762982.363168:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762982.363173:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789762982.363176:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762982.363182:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2917 00000100:00080000:3.0:1789762982.363187:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762982.363191:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762982.363198:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789762982.363215:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565828224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789762982.363230:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789762982.363235:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789762982.363238:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789762982.363243:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2917 00000100:00080000:3.0:1789762982.363247:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789762982.363252:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789762982.363258:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789762982.363273:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565828352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762982.363325:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565828096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762982.363374:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565828224:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762982.363415:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565828352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762982.426902:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762983.450784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762984.474877:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789762985.434922:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789762985.434968:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789762985.434982:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762985.435032:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789762985.435045:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762982/real 1789762985] req@ffff93f844d67800 x1876699565827968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000100:1.0:1789762985.435046:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789762985.435060:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789762985.435086:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565827968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762985.435099:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762985.435105:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762985.435110:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565827968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762985.435130:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762982/real 1789762985] req@ffff93f844d67480 x1876699565828096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762985.435143:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565828096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762985.435152:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762985.435156:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762985.435159:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565828096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789762985.435166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762982/real 1789762985] req@ffff93f844d64700 x1876699565828224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762985.435178:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565828224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762985.435187:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762985.435191:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762985.435194:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565828224:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789762985.435201:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762982/real 1789762985] req@ffff93f844d67b80 x1876699565828352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762985.435214:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565828352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789762985.435224:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789762985.435227:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789762985.435230:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565828352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762985.498750:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762986.522773:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762987.482725:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762987.482745:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762987.482749:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762987.482759:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2922 00000100:00080000:2.0:1789762987.482771:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762987.482778:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762987.482814:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565828480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762987.482839:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762987.482842:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762987.482844:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762987.482847:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2922 00000100:00080000:2.0:1789762987.482850:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762987.482854:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762987.482862:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762987.482869:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565828480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789762987.482874:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565828608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762987.482889:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762987.482891:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762987.482893:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762987.482896:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2922 00000100:00080000:2.0:1789762987.482899:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762987.482901:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762987.482904:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762987.482914:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565828736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762987.482934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762987.482938:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762987.482942:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1789762987.482944:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789762987.482949:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2922 00000100:00080000:2.0:1789762987.482960:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762987.482965:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762987.482970:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762987.482982:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565828864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762987.483014:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762987/real 1789762987] req@ffff93f8517c0000 x1876699565828480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762987.483042:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565828480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762987.483063:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762987.483068:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762987.483073:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565828480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762987.483097:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565828608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762987.483121:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762987.483136:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762987/real 1789762987] req@ffff93f8517c3b80 x1876699565828608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762987.483149:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565828608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762987.483158:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762987.483162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762987.483165:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565828608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762987.483172:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565828736:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762987.483194:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762987.483206:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762987/real 1789762987] req@ffff93f8517c1f80 x1876699565828736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762987.483218:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565828736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762987.483226:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762987.483229:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762987.483233:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565828736:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762987.483240:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565828864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762987.483260:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762987.483272:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762987/real 1789762987] req@ffff93f8517c2300 x1876699565828864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762987.483283:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565828864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762987.483291:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762987.483294:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762987.483299:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565828864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762987.546783:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762988.570774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762989.594799:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762990.618877:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762991.642910:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762992.602840:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762992.602853:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789762992.602860:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762992.602874:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2927 00000100:00080000:0.0:1789762992.602892:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762992.602905:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789762992.602990:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565828992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762992.603036:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762992.603043:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762992.603047:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762992.603056:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2927 00000100:00080000:0.0:1789762992.603067:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762992.603077:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789762992.603112:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789762992.603113:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565828992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789762992.603157:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565829120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762992.603179:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762992.603197:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789762992.603201:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762992.603211:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2927 00000100:00080000:0.0:1789762992.603236:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762992.603244:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1789762992.603253:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1789762992.603254:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789762992.603287:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565829248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789762992.603305:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789762992.603310:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789762992.603314:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789762992.603327:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2927 00000100:00080000:0.0:1789762992.603334:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789762992.603342:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789762992.603342:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762992/real 1789762992] req@ffff93f850d41180 x1876699565828992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1789762992.603370:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789762992.603414:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565829376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762992.603438:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565828992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762992.603471:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762992.603482:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762992.603491:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565828992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762992.603557:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565829120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762992.603599:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762992.603618:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762992/real 1789762992] req@ffff93f850d43100 x1876699565829120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762992.603637:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565829120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762992.603685:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762992.603690:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762992.603695:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565829120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762992.603704:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565829248:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762992.603732:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762992.603746:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762992/real 1789762992] req@ffff93f850d41f80 x1876699565829248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762992.603762:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565829248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762992.603772:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762992.603777:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762992.603781:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565829248:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762992.603788:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565829376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762992.603808:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762992.603820:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762992/real 1789762992] req@ffff93f850d41500 x1876699565829376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762992.603834:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565829376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762992.603853:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762992.603857:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762992.603861:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565829376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762992.666915:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762993.690787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762994.714757:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762995.738801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762996.762864:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762997.722733:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762997.722751:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789762997.722755:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762997.722763:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2932 00000100:00080000:2.0:1789762997.722772:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762997.722779:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789762997.722814:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565829504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762997.722838:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762997.722841:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762997.722843:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762997.722846:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2932 00000100:00080000:2.0:1789762997.722848:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762997.722851:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762997.722859:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762997.722870:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565829632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762997.722874:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565829504:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789762997.722887:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762997.722889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789762997.722891:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762997.722894:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2932 00000100:00080000:2.0:1789762997.722896:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762997.722898:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789762997.722901:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789762997.722909:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565829760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762997.722917:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789762997.722933:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789762997.722935:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789762997.722938:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2932 00000100:00080000:2.0:1789762997.722942:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789762997.722945:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1789762997.722950:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1789762997.722952:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789762997.722964:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565829888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762997.722993:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762997/real 1789762997] req@ffff93f8517c1500 x1876699565829504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762997.723012:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565829504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762997.723021:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762997.723025:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762997.723028:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565829504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789762997.723046:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565829632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762997.723069:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762997.723078:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762997/real 1789762997] req@ffff93f8517c0380 x1876699565829632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762997.723086:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565829632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762997.723092:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762997.723094:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762997.723096:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565829632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789762997.723101:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565829760:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789762997.723121:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762997.723136:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762997/real 1789762997] req@ffff93f8517c0a80 x1876699565829760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762997.723144:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565829760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762997.723151:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762997.723154:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762997.723158:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565829760:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789762997.723162:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565829888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789762997.723175:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789762997.723181:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789762997/real 1789762997] req@ffff93f8517c2680 x1876699565829888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789762997.723191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565829888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789762997.723196:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789762997.723201:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789762997.723205:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565829888:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789762997.786788:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762998.810794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762999.834826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763000.858793:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763001.882773:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763002.842760:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763002.842769:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763002.842773:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763002.842782:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2937 00000100:00080000:3.0:1789763002.842791:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763002.842798:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763002.842844:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565830016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763002.842871:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763002.842874:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763002.842876:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763002.842878:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2937 00000100:00080000:3.0:1789763002.842893:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763002.842896:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763002.842906:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763002.842917:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565830144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763002.842930:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1789763002.842931:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565830016:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763002.842940:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763002.842947:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763002.842952:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2937 00000100:00080000:3.0:1789763002.842954:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763002.842956:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763002.842961:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763002.842973:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565830272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763002.842980:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763002.842983:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763002.842984:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763002.842986:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2937 00000100:00080000:3.0:1789763002.842992:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763002.842995:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763002.842999:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763002.843006:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565830400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763002.843022:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763002.843075:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763002/real 1789763002] req@ffff93f844d66680 x1876699565830016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763002.843093:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565830016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763002.843104:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763002.843109:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763002.843112:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565830016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763002.843138:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565830144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763002.843154:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763002.843163:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763002/real 1789763002] req@ffff93f844d65880 x1876699565830144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763002.843171:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565830144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763002.843177:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763002.843180:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763002.843182:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565830144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763002.843186:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565830272:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763002.843197:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763002.843203:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763002/real 1789763002] req@ffff93f844d67b80 x1876699565830272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763002.843210:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565830272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763002.843220:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763002.843223:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763002.843225:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565830272:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763002.843229:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565830400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763002.843239:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763002.843245:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763002/real 1789763002] req@ffff93f844d64700 x1876699565830400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763002.843252:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565830400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763002.843258:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763002.843260:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763002.843261:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565830400:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763002.906761:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763003.930763:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763004.954765:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763005.978813:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763007.002783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763007.962730:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763007.962738:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763007.962741:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763007.962750:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2942 00000100:00080000:2.0:1789763007.962758:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763007.962763:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763007.962795:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565830528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763007.962818:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763007.962821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763007.962822:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763007.962825:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2942 00000100:00080000:2.0:1789763007.962827:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763007.962831:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763007.962838:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763007.962862:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565830528:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763007.962863:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565830656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763007.962877:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763007.962879:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763007.962880:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763007.962883:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2942 00000100:00080000:2.0:1789763007.962886:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763007.962889:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763007.962893:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763007.962901:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565830784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763007.962908:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763007.962911:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763007.962912:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763007.962914:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2942 00000100:00080000:2.0:1789763007.962917:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763007.962919:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763007.962922:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789763007.962926:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763007.962933:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565830912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763007.962965:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763007/real 1789763007] req@ffff93f8517c2d80 x1876699565830528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763007.962979:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565830528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763007.962987:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763007.962990:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763007.962993:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565830528:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763007.963011:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565830656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763007.963025:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763007.963033:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763007/real 1789763007] req@ffff93f8517c1180 x1876699565830656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763007.963041:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565830656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763007.963047:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763007.963049:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763007.963052:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565830656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763007.963055:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565830784:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763007.963066:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763007.963072:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763007/real 1789763007] req@ffff93f8517c3100 x1876699565830784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763007.963079:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565830784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763007.963084:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763007.963086:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763007.963088:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565830784:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763007.963091:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565830912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763007.963100:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763007.963105:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763007/real 1789763007] req@ffff93f8517c2a00 x1876699565830912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763007.963113:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565830912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763007.963118:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763007.963119:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763007.963121:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565830912:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763008.026764:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763009.050773:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763010.074756:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763011.098801:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763012.122825:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763013.082732:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763013.082743:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763013.082747:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763013.082757:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2948 00000100:00080000:2.0:1789763013.082766:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763013.082774:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763013.082821:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565831040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763013.082846:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763013.082850:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763013.082853:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763013.082857:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2948 00000100:00080000:2.0:1789763013.082861:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763013.082865:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763013.082874:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763013.082880:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565831040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763013.082892:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565831168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763013.082904:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763013.082908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763013.082910:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763013.082921:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2948 00000100:00080000:2.0:1789763013.082928:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763013.082932:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763013.082938:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:1.0:1789763013.082949:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763013.082951:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565831296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763013.082965:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763013.082969:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763013.082971:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763013.082975:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2948 00000100:00080000:2.0:1789763013.082979:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763013.082984:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763013.082989:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763013.082999:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565831424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763013.083007:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763013/real 1789763013] req@ffff93f8517c1880 x1876699565831040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763013.083031:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565831040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763013.083042:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763013.083046:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763013.083049:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565831040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763013.083068:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565831168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763013.083087:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763013.083097:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763013/real 1789763013] req@ffff93f8517c3480 x1876699565831168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763013.083105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565831168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763013.083115:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763013.083118:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763013.083120:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565831168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763013.083123:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565831296:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763013.083134:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763013.083140:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763013/real 1789763013] req@ffff93f8517c3800 x1876699565831296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763013.083147:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565831296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763013.083152:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763013.083154:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763013.083156:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565831296:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763013.083159:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565831424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763013.083169:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763013.083174:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763013/real 1789763013] req@ffff93f8517c0e00 x1876699565831424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763013.083181:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565831424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763013.083186:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763013.083188:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763013.083190:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565831424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763013.146750:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763014.170744:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763015.194823:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763016.218944:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763017.242894:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763018.202855:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763018.202868:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763018.202873:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763018.202884:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2953 00000100:00080000:2.0:1789763018.202897:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763018.202906:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763018.202955:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565831552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763018.202992:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763018.202997:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763018.203000:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763018.203004:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2953 00000100:00080000:2.0:1789763018.203008:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763018.203012:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763018.203023:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763018.203038:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565831680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763018.203052:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763018.203057:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763018.203060:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763018.203066:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2953 00000100:00080000:2.0:1789763018.203069:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763018.203073:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763018.203082:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763018.203096:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565831808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763018.203097:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565831552:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763018.203141:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763018.203148:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763018.203152:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763018.203164:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2953 00000100:00080000:2.0:1789763018.203173:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763018.203181:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763018.203193:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763018.203219:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565831936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763018.203233:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763018.203312:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763018/real 1789763018] req@ffff93f8517c0e00 x1876699565831552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763018.203340:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565831552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763018.203365:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763018.203372:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763018.203378:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565831552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763018.203389:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565831680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763018.203418:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763018.203434:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763018/real 1789763018] req@ffff93f8517c3800 x1876699565831680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763018.203450:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565831680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763018.203460:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763018.203464:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763018.203468:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565831680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763018.203504:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565831808:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763018.203528:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763018.203547:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763018/real 1789763018] req@ffff93f8517c3480 x1876699565831808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763018.203562:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565831808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763018.203572:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763018.203575:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763018.203579:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565831808:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763018.203586:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565831936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763018.203607:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763018.203629:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763018/real 1789763018] req@ffff93f8517c1880 x1876699565831936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763018.203678:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565831936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763018.203690:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763018.203694:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763018.203698:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565831936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763018.266870:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763019.290931:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763020.314945:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763021.338890:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763022.362921:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763023.322856:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763023.322869:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763023.322876:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763023.322889:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2958 00000100:00080000:2.0:1789763023.322905:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763023.322926:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763023.322972:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565832064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763023.323011:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763023.323018:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763023.323022:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763023.323030:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2958 00000100:00080000:2.0:1789763023.323039:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763023.323048:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763023.323064:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763023.323090:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565832064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763023.323095:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565832192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763023.323145:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763023.323149:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763023.323154:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763023.323161:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2958 00000100:00080000:2.0:1789763023.323167:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763023.323173:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763023.323183:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763023.323204:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565832320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763023.323217:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763023.323222:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763023.323225:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763023.323231:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2958 00000100:00080000:2.0:1789763023.323235:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763023.323240:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763023.323245:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789763023.323261:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763023.323272:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565832448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763023.323374:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763023/real 1789763023] req@ffff93f8517c2a00 x1876699565832064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763023.323417:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565832064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763023.323437:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763023.323444:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763023.323450:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565832064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763023.323491:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565832192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763023.323541:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763023.323570:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763023/real 1789763023] req@ffff93f8517c3100 x1876699565832192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763023.323598:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565832192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763023.323611:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763023.323616:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763023.323621:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565832192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763023.323631:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565832320:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763023.323700:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763023.323717:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763023/real 1789763023] req@ffff93f8517c1180 x1876699565832320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763023.323759:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565832320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763023.323781:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763023.323786:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763023.323790:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565832320:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763023.323800:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565832448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763023.323831:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763023.323846:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763023/real 1789763023] req@ffff93f8517c2d80 x1876699565832448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763023.323862:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565832448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763023.323872:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763023.323877:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763023.323881:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565832448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763023.386916:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763024.410897:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763025.434840:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763026.458905:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763027.482907:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763028.442784:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763028.442796:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763028.442800:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763028.442814:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2963 00000100:00080000:2.0:1789763028.442828:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763028.442837:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763028.442881:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565832576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763028.442911:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763028.442916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763028.442918:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763028.442923:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2963 00000100:00080000:2.0:1789763028.442928:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763028.442934:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763028.442944:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763028.442956:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565832576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763028.442962:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565832704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763028.442978:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763028.442983:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763028.442986:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763028.442991:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2963 00000100:00080000:2.0:1789763028.442996:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763028.443001:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763028.443006:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763028.443021:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565832832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763028.443033:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:1.0:1789763028.443040:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763028.443041:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763028.443047:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763028.443053:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2963 00000100:00080000:2.0:1789763028.443059:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763028.443064:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763028.443070:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763028.443085:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565832960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763028.443098:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763028/real 1789763028] req@ffff93f8517c2680 x1876699565832576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763028.443125:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565832576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763028.443139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763028.443145:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763028.443148:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565832576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763028.443175:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565832704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763028.443197:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763028.443207:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763028/real 1789763028] req@ffff93f8517c0a80 x1876699565832704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763028.443218:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565832704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763028.443225:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763028.443229:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763028.443233:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565832704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763028.443241:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565832832:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763028.443265:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763028.443278:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763028/real 1789763028] req@ffff93f8517c0380 x1876699565832832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763028.443294:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565832832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763028.443304:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763028.443317:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763028.443323:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565832832:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763028.443329:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565832960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763028.443347:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763028.443356:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763028/real 1789763028] req@ffff93f8517c1500 x1876699565832960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763028.443365:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565832960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763028.443377:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763028.443379:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763028.443382:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565832960:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763028.506903:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763029.530854:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763030.554909:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763031.578937:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763032.602861:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763033.562819:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763033.562832:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763033.562838:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763033.562855:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2968 00000100:00080000:2.0:1789763033.562872:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763033.562884:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763033.562945:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565833088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763033.563002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763033.563011:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763033.563017:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763033.563030:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2968 00000100:00080000:2.0:1789763033.563041:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763033.563053:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763033.563059:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565833088:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789763033.563073:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763033.563113:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565833216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763033.563134:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763033.563140:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763033.563143:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763033.563153:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2968 00000800:00000100:3.0:1789763033.563155:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763033.563165:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763033.563179:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763033.563192:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763033.563220:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763033/real 1789763033] req@ffff93f8517c2300 x1876699565833088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763033.563231:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565833344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763033.563266:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565833088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763033.563268:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763033.563280:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763033.563285:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763033.563297:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2968 00000100:00080000:2.0:1789763033.563306:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763033.563315:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763033.563327:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763033.563352:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565833472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763033.563605:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763033.563616:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763033.563626:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565833088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763033.563842:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565833216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763033.563886:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763033.563907:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763033/real 1789763033] req@ffff93f8517c1f80 x1876699565833216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763033.563924:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565833216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763033.563936:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763033.563940:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763033.563944:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565833216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763033.563955:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565833344:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763033.563977:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763033.563989:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763033/real 1789763033] req@ffff93f8517c3b80 x1876699565833344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763033.564003:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565833344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763033.564013:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763033.564017:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763033.564021:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565833344:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763033.564028:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565833472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763033.564047:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763033.564058:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763033/real 1789763033] req@ffff93f8517c0000 x1876699565833472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763033.564072:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565833472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763033.564081:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763033.564084:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763033.564088:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565833472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763033.626901:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763034.650853:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763035.674857:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763036.698814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763037.722852:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763038.682767:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763038.682780:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763038.682786:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763038.682801:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2973 00000100:00080000:2.0:1789763038.682815:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763038.682825:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763038.682876:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565833600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763038.682915:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763038.682920:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763038.682936:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763038.682942:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2973 00000100:00080000:2.0:1789763038.682947:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763038.682953:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763038.682965:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763038.682986:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565833728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763038.682997:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565833600:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763038.683016:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763038.683023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763038.683026:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763038.683032:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2973 00000100:00080000:2.0:1789763038.683038:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763038.683043:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763038.683052:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763038.683080:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565833856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763038.683100:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763038.683108:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763038.683115:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763038.683124:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2973 00000800:00000100:3.0:1789763038.683134:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763038.683135:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763038.683149:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763038.683161:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763038.683192:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565833984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763038.683234:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763038/real 1789763038] req@ffff93f8517c0000 x1876699565833600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763038.683265:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565833600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763038.683283:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763038.683289:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763038.683295:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565833600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763038.683337:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565833728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763038.683369:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763038.683386:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763038/real 1789763038] req@ffff93f8517c3b80 x1876699565833728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763038.683402:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565833728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763038.683413:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763038.683419:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763038.683426:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565833728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763038.683439:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565833856:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763038.683481:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763038.683505:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763038/real 1789763038] req@ffff93f8517c1f80 x1876699565833856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763038.683532:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565833856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763038.683547:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763038.683551:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763038.683556:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565833856:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763038.683564:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565833984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763038.683597:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763038.683611:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763038/real 1789763038] req@ffff93f8517c2300 x1876699565833984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763038.683693:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565833984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763038.683704:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763038.683709:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763038.683713:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565833984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763038.746854:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763039.770848:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763040.794867:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763041.818939:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763042.842856:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763043.802807:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763043.802822:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763043.802829:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763043.802845:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2978 00000100:00080000:2.0:1789763043.802861:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763043.802873:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763043.802926:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565834112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763043.802962:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763043.802967:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763043.802969:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763043.802974:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2978 00000100:00080000:2.0:1789763043.802979:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763043.802984:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763043.802995:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763043.803014:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565834240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763043.803022:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565834112:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763043.803036:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763043.803041:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763043.803043:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763043.803048:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2978 00000100:00080000:2.0:1789763043.803051:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763043.803055:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763043.803061:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763043.803074:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565834368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763043.803087:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763043.803091:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763043.803102:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763043.803106:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2978 00000100:00080000:2.0:1789763043.803110:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763043.803115:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763043.803120:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763043.803135:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565834496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763043.803139:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763043.803226:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763043/real 1789763043] req@ffff93f8517c1500 x1876699565834112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763043.803272:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565834112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763043.803292:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763043.803298:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763043.803303:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565834112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763043.803331:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565834240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763043.803363:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763043.803379:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763043/real 1789763043] req@ffff93f8517c0380 x1876699565834240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763043.803393:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565834240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763043.803403:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763043.803417:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763043.803421:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565834240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763043.803427:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565834368:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763043.803444:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763043.803454:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763043/real 1789763043] req@ffff93f8517c0a80 x1876699565834368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763043.803467:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565834368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763043.803483:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763043.803486:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763043.803490:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565834368:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763043.803495:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565834496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763043.803511:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763043.803521:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763043/real 1789763043] req@ffff93f8517c2680 x1876699565834496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763043.803532:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565834496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763043.803541:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763043.803544:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763043.803547:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565834496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763043.866865:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763044.890807:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763045.914833:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763046.938799:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763047.962883:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763048.923052:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763048.923064:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763048.923070:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763048.923082:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2983 00000100:00080000:3.0:1789763048.923095:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763048.923104:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763048.923166:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565834624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763048.923210:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763048.923215:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763048.923218:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763048.923223:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2983 00000100:00080000:3.0:1789763048.923227:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763048.923232:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763048.923244:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763048.923260:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565834752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763048.923275:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565834624:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763048.923280:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763048.923293:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763048.923295:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763048.923302:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2983 00000100:00080000:3.0:1789763048.923306:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763048.923311:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763048.923318:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763048.923336:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565834880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763048.923349:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763048.923354:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763048.923357:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763048.923363:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2983 00000100:00080000:3.0:1789763048.923369:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763048.923373:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763048.923380:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763048.923394:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565835008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763048.923445:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565834752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763048.923494:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565834880:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763048.923520:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565835008:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763048.986871:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763050.010861:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763051.034792:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789763051.995090:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789763051.995188:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789763051.995233:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789763051.995301:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789763051.995327:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789763051.995350:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789763051.995382:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763048/real 1789763051] req@ffff93f844d67800 x1876699565834752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763051.995450:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565834752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763051.995485:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763051.995494:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763051.995503:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565834752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763051.995518:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763048/real 1789763051] req@ffff93f844d67100 x1876699565834880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763051.995577:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565834880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763051.995594:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763051.995602:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763051.995609:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565834880:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763051.995624:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763048/real 1789763051] req@ffff93f844d66a00 x1876699565835008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763051.995690:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565835008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763051.995709:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763051.995719:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763051.995729:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565835008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763051.995746:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763048/real 1789763051] req@ffff93f844d67480 x1876699565834624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763051.995774:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565834624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763051.995792:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763051.995800:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763051.995809:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565834624:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789763052.058882:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763053.082856:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763054.042791:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763054.042806:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763054.042813:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763054.042832:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2989 00000100:00080000:3.0:1789763054.042852:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763054.042867:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763054.042965:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565835136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763054.043019:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763054.043026:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763054.043030:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763054.043038:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2989 00000100:00080000:3.0:1789763054.043047:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763054.043057:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763054.043078:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763054.043106:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565835264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763054.043114:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565835136:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763054.043140:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763054.043148:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763054.043151:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763054.043161:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2989 00000100:00080000:3.0:1789763054.043169:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763054.043178:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763054.043189:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763054.043215:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565835392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763054.043240:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763054.043257:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763054.043266:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763054.043270:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763054.043280:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2989 00000100:00080000:3.0:1789763054.043287:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763054.043295:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763054.043316:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763054.043325:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763054/real 1789763054] req@ffff93f844d65c00 x1876699565835136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763054.043346:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565835520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763054.043359:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565835136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763054.043394:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763054.043404:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763054.043415:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565835136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763054.043467:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565835264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763054.043543:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763054.043572:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763054/real 1789763054] req@ffff93f844d64e00 x1876699565835264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763054.043598:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565835264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763054.043615:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763054.043622:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763054.043628:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565835264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763054.043682:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565835392:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763054.043730:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763054.043756:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763054/real 1789763054] req@ffff93f844d66d80 x1876699565835392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763054.043780:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565835392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763054.043797:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763054.043803:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763054.043810:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565835392:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763054.043824:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565835520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763054.043864:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763054.043888:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763054/real 1789763054] req@ffff93f844d65500 x1876699565835520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763054.043911:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565835520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763054.043927:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763054.043932:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763054.043938:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565835520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763054.106891:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763055.130952:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763056.154847:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763057.178809:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763058.202842:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763059.162830:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763059.162846:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763059.162853:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763059.162870:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2994 00000100:00080000:0.0:1789763059.162887:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763059.162899:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763059.162981:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565835648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763059.163024:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763059.163030:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763059.163033:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763059.163038:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2994 00000100:00080000:0.0:1789763059.163043:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763059.163048:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763059.163062:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763059.163078:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565835776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763059.163091:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763059.163096:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763059.163098:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763059.163105:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2994 00000100:00080000:0.0:1789763059.163110:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763059.163119:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763059.163125:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763059.163139:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565835904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763059.163154:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763059.163159:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763059.163163:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763059.163174:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2994 00000100:00100000:1.0:1789763059.163178:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565835648:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763059.163179:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763059.163192:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763059.163200:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763059.163211:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565836032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763059.163328:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763059.163401:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763059/real 1789763059] req@ffff93f850d43800 x1876699565835648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763059.163429:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565835648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763059.163445:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763059.163451:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763059.163456:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565835648:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763059.163494:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565835776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763059.163530:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763059.163548:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763059/real 1789763059] req@ffff93f850d43480 x1876699565835776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763059.163565:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565835776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763059.163577:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763059.163581:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763059.163585:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565835776:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763059.163616:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565835904:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763059.163639:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763059.163693:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763059/real 1789763059] req@ffff93f850d41500 x1876699565835904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763059.163709:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565835904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763059.163720:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763059.163724:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763059.163728:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565835904:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763059.163735:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565836032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763059.163760:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763059.163773:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763059/real 1789763059] req@ffff93f850d41f80 x1876699565836032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763059.163788:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565836032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763059.163806:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763059.163810:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763059.163814:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565836032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763059.226918:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763060.250881:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763061.274882:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763062.298882:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763063.322875:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763064.282850:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763064.282872:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763064.282881:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763064.282902:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 2999 00000100:00080000:0.0:1789763064.282921:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763064.282936:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763064.283015:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565836160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763064.283064:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763064.283073:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763064.283077:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763064.283086:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2999 00000100:00080000:0.0:1789763064.283094:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763064.283103:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763064.283123:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763064.283154:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565836288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763064.283179:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1789763064.283186:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565836160:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763064.283187:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763064.283194:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763064.283208:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 2999 00000100:00080000:0.0:1789763064.283214:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763064.283219:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763064.283227:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763064.283246:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565836416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763064.283260:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763064.283266:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763064.283268:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763064.283273:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 2999 00000100:00080000:0.0:1789763064.283278:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763064.283282:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763064.283290:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763064.283313:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565836544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763064.283342:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763064.283410:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763064/real 1789763064] req@ffff93f850d43100 x1876699565836160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763064.283472:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565836160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763064.283488:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763064.283494:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763064.283507:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565836160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763064.283554:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565836288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763064.283585:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763064.283602:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763064/real 1789763064] req@ffff93f850d41180 x1876699565836288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763064.283617:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565836288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763064.283628:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763064.283633:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763064.283637:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565836288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763064.283679:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565836416:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763064.283706:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763064.283721:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763064/real 1789763064] req@ffff93f850d41c00 x1876699565836416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763064.283736:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565836416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763064.283747:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763064.283751:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763064.283755:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565836416:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763064.283761:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565836544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763064.283781:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763064.283792:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763064/real 1789763064] req@ffff93f850d42680 x1876699565836544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763064.283817:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565836544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763064.283827:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763064.283830:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763064.283833:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565836544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763064.346913:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763065.370990:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763066.394820:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763067.418867:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763068.442767:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763069.402893:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763069.402911:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763069.402919:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763069.402935:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3004 00000100:00080000:0.0:1789763069.402951:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763069.402963:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763069.403033:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565836672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763069.403075:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763069.403081:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763069.403084:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763069.403089:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3004 00000100:00080000:0.0:1789763069.403093:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763069.403098:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763069.403112:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763069.403129:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565836800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763069.403143:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763069.403148:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763069.403151:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763069.403156:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3004 00000100:00080000:0.0:1789763069.403162:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763069.403179:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763069.403191:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763069.403191:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565836672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763069.403210:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565836928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763069.403226:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763069.403230:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763069.403241:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763069.403246:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3004 00000100:00080000:0.0:1789763069.403250:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763069.403255:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763069.403261:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763069.403277:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565837056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763069.403349:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763069.403472:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763069/real 1789763069] req@ffff93f850d40e00 x1876699565836672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763069.403523:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565836672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763069.403546:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763069.403556:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763069.403565:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565836672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763069.403586:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565836800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763069.403638:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763069.403706:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763069/real 1789763069] req@ffff93f850d40000 x1876699565836800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763069.403724:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565836800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763069.403741:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763069.403745:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763069.403749:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565836800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763069.403788:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565836928:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763069.403835:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763069.403888:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763069/real 1789763069] req@ffff93f850d42300 x1876699565836928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763069.403915:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565836928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763069.403935:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763069.403939:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763069.403944:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565836928:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763069.403953:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565837056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763069.403992:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763069.404009:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763069/real 1789763069] req@ffff93f850d40380 x1876699565837056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763069.404026:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565837056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763069.404037:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763069.404042:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763069.404046:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565837056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763069.466891:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763070.490932:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763071.514877:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763072.538901:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763073.562860:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763074.522860:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763074.522871:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763074.522877:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763074.522890:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3009 00000100:00080000:3.0:1789763074.522904:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763074.522914:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763074.522973:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565837184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763074.523013:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763074.523018:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763074.523020:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763074.523038:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3009 00000100:00080000:3.0:1789763074.523043:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763074.523049:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763074.523061:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763074.523080:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565837312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763074.523089:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565837184:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763074.523114:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763074.523118:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763074.523122:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763074.523127:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3009 00000100:00080000:3.0:1789763074.523132:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763074.523136:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763074.523143:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763074.523168:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565837440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763074.523181:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763074.523186:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763074.523188:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763074.523191:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3009 00000100:00080000:3.0:1789763074.523198:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763074.523202:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763074.523210:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789763074.523218:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763074.523242:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565837568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763074.523318:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763074/real 1789763074] req@ffff93f844d64380 x1876699565837184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763074.523391:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565837184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763074.523419:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763074.523427:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763074.523435:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565837184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763074.523482:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565837312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763074.523520:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763074.523540:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763074/real 1789763074] req@ffff93f844d64000 x1876699565837312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763074.523556:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565837312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763074.523576:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763074.523580:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763074.523585:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565837312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763074.523591:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565837440:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763074.523612:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763074.523624:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763074/real 1789763074] req@ffff93f844d64a80 x1876699565837440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763074.523638:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565837440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763074.523685:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763074.523689:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763074.523693:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565837440:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763074.523701:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565837568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763074.523726:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763074.523740:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763074/real 1789763074] req@ffff93f844d65f80 x1876699565837568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763074.523767:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565837568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763074.523777:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763074.523781:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763074.523785:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565837568:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763074.586895:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763075.610765:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763076.634889:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763077.658860:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763078.682886:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763079.642795:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763079.642812:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763079.642819:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763079.642835:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3014 00000100:00080000:3.0:1789763079.642863:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763079.642875:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763079.642931:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565837696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763079.642971:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763079.642976:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763079.642979:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763079.642985:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3014 00000100:00080000:3.0:1789763079.642990:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763079.642995:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763079.643008:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763079.643038:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565837696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763079.643041:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565837824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763079.643078:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763079.643083:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763079.643088:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763079.643107:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3014 00000100:00080000:3.0:1789763079.643112:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763079.643117:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763079.643125:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763079.643140:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565837952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763079.643184:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763079.643195:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763079.643201:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763079.643204:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763079.643213:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3014 00000100:00080000:3.0:1789763079.643221:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763079.643227:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763079.643233:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763079.643247:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565838080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763079.643275:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763079/real 1789763079] req@ffff93f844d65180 x1876699565837696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763079.643303:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565837696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763079.643314:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763079.643336:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763079.643341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565837696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763079.643367:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565837824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763079.643390:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763079.643402:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763079/real 1789763079] req@ffff93f844d66300 x1876699565837824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763079.643412:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565837824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763079.643424:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763079.643427:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763079.643430:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565837824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763079.643435:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565837952:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763079.643452:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763079.643465:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763079/real 1789763079] req@ffff93f844d65f80 x1876699565837952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763079.643480:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565837952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763079.643490:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763079.643495:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763079.643499:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565837952:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763079.643507:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565838080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763079.643529:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763079.643539:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763079/real 1789763079] req@ffff93f844d64a80 x1876699565838080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763079.643548:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565838080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763079.643554:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763079.643557:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763079.643560:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565838080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763079.706920:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763080.730875:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763081.754857:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763082.778854:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763083.802881:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763084.762874:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763084.762889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763084.762895:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763084.762911:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3019 00000100:00080000:0.0:1789763084.762926:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763084.762938:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763084.762997:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565838208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763084.763041:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763084.763049:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763084.763052:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763084.763059:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3019 00000100:00080000:0.0:1789763084.763064:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763084.763071:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763084.763098:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565838208:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1789763084.763100:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763084.763134:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565838336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763084.763154:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763084.763159:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763084.763163:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763084.763169:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3019 00000100:00080000:0.0:1789763084.763174:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763084.763190:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763084.763197:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:1.0:1789763084.763202:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763084.763228:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565838464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763084.763246:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763084.763251:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763084.763253:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763084.763260:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3019 00000100:00100000:1.0:1789763084.763260:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763084/real 1789763084] req@ffff93f850d43b80 x1876699565838208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763084.763274:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763084.763309:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763084.763318:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763084.763320:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565838208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763084.763351:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565838592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763084.763356:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763084.763380:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763084.763398:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565838208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763084.763429:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565838336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763084.763454:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763084.763465:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763084/real 1789763084] req@ffff93f850d40700 x1876699565838336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763084.763474:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565838336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763084.763481:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763084.763483:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763084.763486:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565838336:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763084.763491:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565838464:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763084.763503:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763084.763511:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763084/real 1789763084] req@ffff93f850d40380 x1876699565838464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763084.763519:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565838464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763084.763525:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763084.763527:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763084.763530:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565838464:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763084.763534:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565838592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763084.763546:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763084.763553:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763084/real 1789763084] req@ffff93f850d42300 x1876699565838592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763084.763561:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565838592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763084.763583:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763084.763586:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763084.763588:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565838592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763084.826838:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763085.850843:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763086.874833:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763087.898861:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763088.922871:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763089.882843:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763089.882862:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763089.882870:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763089.882891:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3024 00000100:00080000:1.0:1789763089.882911:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763089.882926:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763089.882993:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565838720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763089.883033:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763089.883038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763089.883041:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763089.883047:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3024 00000100:00080000:1.0:1789763089.883053:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763089.883061:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763089.883076:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763089.883088:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565838720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763089.883099:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565838848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763089.883116:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763089.883120:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763089.883123:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763089.883127:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3024 00000100:00080000:1.0:1789763089.883132:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763089.883136:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763089.883143:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763089.883158:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565838976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763089.883171:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763089.883175:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763089.883178:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763089.883184:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3024 00000100:00080000:1.0:1789763089.883201:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763089.883206:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763089.883213:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789763089.883215:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763089.883237:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565839104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763089.883326:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763089/real 1789763089] req@ffff93f844ccb800 x1876699565838720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763089.883359:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565838720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763089.883375:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763089.883380:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763089.883386:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565838720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763089.883421:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565838848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763089.883454:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763089.883470:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763089/real 1789763089] req@ffff93f844cc8700 x1876699565838848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763089.883487:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565838848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763089.883498:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763089.883502:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763089.883506:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565838848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763089.883513:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565838976:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763089.883535:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763089.883546:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763089/real 1789763089] req@ffff93f844ccbb80 x1876699565838976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763089.883570:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565838976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763089.883580:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763089.883584:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763089.883587:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565838976:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763089.883594:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565839104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763089.883616:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763089.883627:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763089/real 1789763089] req@ffff93f844cc8000 x1876699565839104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763089.883696:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565839104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763089.883707:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763089.883711:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763089.883715:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565839104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763089.946930:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763090.970831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763091.994829:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763093.018815:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763094.042775:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763095.002897:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763095.002920:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763095.002928:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763095.002951:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3029 00000100:00080000:1.0:1789763095.002972:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763095.002985:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763095.003063:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565839232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763095.003122:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763095.003129:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763095.003134:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763095.003143:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3029 00000100:00080000:1.0:1789763095.003150:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763095.003158:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763095.003178:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763095.003236:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565839360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763095.003257:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763095.003265:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763095.003269:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763095.003276:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3029 00000100:00080000:1.0:1789763095.003284:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763095.003291:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763095.003301:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763095.003323:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565839488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763095.003344:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763095.003350:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763095.003354:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763095.003362:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3029 00000100:00080000:1.0:1789763095.003370:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763095.003377:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763095.003386:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763095.003406:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565839616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763095.003707:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565839232:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1789763095.003863:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763095.003959:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763095/real 1789763095] req@ffff93f844cc8380 x1876699565839232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763095.003990:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565839232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763095.004008:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763095.004015:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763095.004022:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565839232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763095.004034:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565839360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763095.004069:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763095.004087:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763095/real 1789763095] req@ffff93f844cca300 x1876699565839360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763095.004103:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565839360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763095.004114:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763095.004118:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763095.004122:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565839360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763095.004129:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565839488:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763095.004152:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763095.004163:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763095/real 1789763095] req@ffff93f844cc9f80 x1876699565839488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763095.004183:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565839488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763095.004236:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763095.004245:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763095.004252:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565839488:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763095.004265:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565839616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763095.004306:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763095.004323:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763095/real 1789763095] req@ffff93f844cc9880 x1876699565839616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763095.004339:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565839616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763095.004350:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763095.004354:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763095.004358:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565839616:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763095.066807:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763096.090885:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763097.114842:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763098.138865:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763099.162888:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763100.122816:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763100.122833:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763100.122840:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763100.122857:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3035 00000100:00080000:1.0:1789763100.122876:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763100.122890:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763100.122957:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565839744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763100.123001:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763100.123006:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763100.123009:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763100.123014:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3035 00000100:00080000:1.0:1789763100.123019:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763100.123024:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763100.123038:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763100.123057:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565839872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763100.123073:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763100.123078:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763100.123082:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763100.123097:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3035 00000100:00100000:3.0:1789763100.123099:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565839744:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763100.123102:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763100.123117:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763100.123131:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763100.123162:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565840000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763100.123224:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763100.123231:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763100.123250:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763100.123262:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3035 00000800:00000100:3.0:1789763100.123262:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763100.123285:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763100.123298:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763100.123308:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763100.123332:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565840128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763100.123349:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763100/real 1789763100] req@ffff93f844cc9180 x1876699565839744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763100.123382:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565839744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763100.123400:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763100.123406:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763100.123411:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565839744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763100.123445:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565839872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763100.123477:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763100.123493:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763100/real 1789763100] req@ffff93f844cc8a80 x1876699565839872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763100.123509:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565839872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763100.123519:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763100.123524:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763100.123528:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565839872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763100.123535:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565840000:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763100.123555:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763100.123567:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763100/real 1789763100] req@ffff93f844cc9880 x1876699565840000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763100.123581:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565840000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763100.123591:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763100.123595:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763100.123599:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565840000:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763100.123607:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565840128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763100.123627:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763100.123638:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763100/real 1789763100] req@ffff93f844cc9f80 x1876699565840128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763100.123653:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565840128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763100.123721:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763100.123725:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763100.123729:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565840128:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763100.186892:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763101.210908:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763102.234914:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763103.258893:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763104.282958:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763105.242931:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763105.242946:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763105.242954:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763105.242971:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3040 00000100:00080000:2.0:1789763105.242991:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763105.243003:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763105.243075:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565840256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763105.243126:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763105.243158:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763105.243162:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763105.243169:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3040 00000100:00080000:2.0:1789763105.243177:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763105.243185:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763105.243196:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565840256:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789763105.243207:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763105.243250:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565840384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763105.243279:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763105.243288:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763105.243293:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763105.243306:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3040 00000100:00080000:2.0:1789763105.243319:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1789763105.243321:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763105.243341:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763105.243357:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763105.243379:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565840512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763105.243395:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763105.243401:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763105.243404:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789763105.243411:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763105/real 1789763105] req@ffff93f8517c2d80 x1876699565840256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763105.243416:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3040 00000100:00080000:2.0:1789763105.243429:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763105.243441:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763105.243449:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763105.243457:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565840256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763105.243473:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565840640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763105.243481:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763105.243499:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763105.243511:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565840256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763105.243568:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565840384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763105.243602:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763105.243627:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763105/real 1789763105] req@ffff93f8517c1180 x1876699565840384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763105.243695:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565840384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763105.243715:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763105.243721:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763105.243728:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565840384:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763105.243742:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565840512:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763105.243786:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763105.243802:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763105/real 1789763105] req@ffff93f8517c3100 x1876699565840512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763105.243817:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565840512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763105.243828:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763105.243832:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763105.243836:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565840512:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763105.243843:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565840640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763105.243863:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763105.243874:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763105/real 1789763105] req@ffff93f8517c2a00 x1876699565840640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763105.243888:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565840640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763105.243898:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763105.243901:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763105.243905:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565840640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763105.306864:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763106.330907:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763107.354808:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763108.378799:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763109.402869:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763110.362926:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763110.362936:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763110.362940:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763110.362950:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3045 00000100:00080000:3.0:1789763110.362959:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763110.362965:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763110.363004:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565840768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763110.363030:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763110.363033:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763110.363035:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763110.363038:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3045 00000100:00080000:3.0:1789763110.363040:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763110.363043:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763110.363052:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763110.363062:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565840896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763110.363072:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763110.363075:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763110.363078:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789763110.363081:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565840768:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763110.363082:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3045 00000100:00080000:3.0:1789763110.363088:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763110.363109:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763110.363114:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763110.363125:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565841024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763110.363135:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763110.363138:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763110.363140:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763110.363144:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3045 00000100:00080000:3.0:1789763110.363147:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763110.363150:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763110.363155:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763110.363167:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565841152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763110.363168:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763110.363213:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763110/real 1789763110] req@ffff93f844d64000 x1876699565840768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763110.363228:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565840768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763110.363237:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763110.363240:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763110.363243:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565840768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763110.363264:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565840896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763110.363282:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763110.363291:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763110/real 1789763110] req@ffff93f844d64380 x1876699565840896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763110.363301:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565840896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763110.363307:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763110.363310:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763110.363312:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565840896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763110.363317:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565841024:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763110.363336:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763110.363349:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763110/real 1789763110] req@ffff93f844d65500 x1876699565841024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763110.363373:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565841024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763110.363384:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763110.363388:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763110.363391:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565841024:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763110.363396:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565841152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763110.363416:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763110.363430:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763110/real 1789763110] req@ffff93f844d66d80 x1876699565841152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763110.363446:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565841152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763110.363456:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763110.363460:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763110.363464:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565841152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763110.426877:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763111.450874:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763112.474738:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763113.498789:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763114.522876:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763115.482808:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763115.482827:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763115.482835:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763115.482850:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3050 00000100:00080000:3.0:1789763115.482863:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763115.482873:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763115.482940:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565841280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763115.482975:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763115.482980:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763115.482982:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763115.482987:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3050 00000100:00080000:3.0:1789763115.482992:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763115.482997:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763115.483009:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763115.483026:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565841408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763115.483038:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565841280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763115.483084:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763115.483089:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763115.483092:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763115.483099:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3050 00000100:00080000:3.0:1789763115.483105:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763115.483111:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763115.483119:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763115.483137:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565841536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763115.483151:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763115.483156:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763115.483159:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763115.483163:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3050 00000100:00080000:3.0:1789763115.483168:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763115.483173:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763115.483178:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763115.483191:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565841664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763115.483246:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565841408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763115.483300:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565841536:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763115.483333:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565841664:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763115.546901:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763116.570951:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763117.594843:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1789763118.555001:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1789763118.555083:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1789763118.555104:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789763118.555151:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789763118.555162:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789763118.555176:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1789763118.555235:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763115/real 1789763118] req@ffff93f844d65c00 x1876699565841408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763118.555297:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565841408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763118.555319:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763118.555328:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763118.555337:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565841408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763118.555353:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763115/real 1789763118] req@ffff93f844d67480 x1876699565841536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763118.555379:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565841536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763118.555393:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763118.555400:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763118.555407:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565841536:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763118.555420:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763115/real 1789763118] req@ffff93f844d66a00 x1876699565841664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763118.555445:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565841664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763118.555462:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763118.555470:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763118.555477:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565841664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763118.555489:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763115/real 1789763118] req@ffff93f844d64e00 x1876699565841280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763118.555513:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565841280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763118.555529:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763118.555536:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763118.555555:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565841280:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1789763118.618894:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763119.642936:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763120.602789:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763120.602834:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763120.602843:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763120.602872:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3055 00000100:00080000:3.0:1789763120.602888:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763120.602899:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763120.602982:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565841792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763120.603039:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763120.603045:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763120.603048:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763120.603053:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3055 00000100:00080000:3.0:1789763120.603059:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763120.603066:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763120.603086:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763120.603103:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565841920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763120.603110:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565841792:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763120.603132:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763120.603137:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763120.603140:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763120.603147:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3055 00000100:00080000:3.0:1789763120.603153:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763120.603159:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763120.603168:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763120.603186:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565842048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763120.603202:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763120.603220:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763120.603226:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763120.603229:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763120.603236:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3055 00000100:00080000:3.0:1789763120.603242:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763120.603248:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763120.603255:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763120.603272:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565842176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763120.603304:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763120/real 1789763120] req@ffff93f844d67100 x1876699565841792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763120.603334:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565841792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763120.603350:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763120.603357:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763120.603363:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565841792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763120.603394:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565841920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763120.603440:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763120.603461:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763120/real 1789763120] req@ffff93f844d67800 x1876699565841920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763120.603491:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565841920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763120.603504:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763120.603509:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763120.603513:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565841920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763120.603522:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565842048:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763120.603554:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763120.603572:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763120/real 1789763120] req@ffff93f844d64700 x1876699565842048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763120.603588:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565842048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763120.603599:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763120.603603:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763120.603607:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565842048:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763120.603615:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565842176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763120.603687:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763120.603707:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763120/real 1789763120] req@ffff93f844d67b80 x1876699565842176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763120.603725:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565842176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763120.603746:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763120.603752:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763120.603757:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565842176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763120.666804:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763121.690789:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763122.714907:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763123.738881:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763124.762862:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763125.722735:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763125.722745:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763125.722750:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763125.722759:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3060 00000100:00080000:3.0:1789763125.722771:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763125.722779:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763125.722836:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565842304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763125.722868:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763125.722871:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763125.722873:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763125.722878:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3060 00000100:00080000:3.0:1789763125.722883:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763125.722899:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763125.722909:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763125.722915:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565842304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763125.722928:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565842432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763125.722940:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763125.722943:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763125.722945:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763125.722949:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3060 00000100:00080000:3.0:1789763125.722953:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763125.722956:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763125.722960:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763125.723000:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565842560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763125.723011:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763125.723018:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763125.723023:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763125.723025:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763125.723031:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3060 00000100:00080000:3.0:1789763125.723036:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763125.723039:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763125.723045:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763125.723056:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565842688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763125.723066:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763125/real 1789763125] req@ffff93f844d65880 x1876699565842304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763125.723090:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565842304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763125.723101:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763125.723105:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763125.723110:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565842304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763125.723133:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565842432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763125.723154:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763125.723165:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763125/real 1789763125] req@ffff93f844d66680 x1876699565842432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763125.723176:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565842432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763125.723184:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763125.723187:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763125.723190:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565842432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763125.723195:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565842560:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763125.723213:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763125.723228:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763125/real 1789763125] req@ffff93f844d67b80 x1876699565842560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763125.723243:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565842560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763125.723255:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763125.723260:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763125.723265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565842560:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763125.723273:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565842688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763125.723297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763125.723308:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763125/real 1789763125] req@ffff93f844d64700 x1876699565842688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763125.723329:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565842688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763125.723337:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763125.723340:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763125.723342:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565842688:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763125.786787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763126.810850:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763127.834902:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763128.858791:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763129.882883:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763130.842821:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763130.842831:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763130.842836:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763130.842845:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3065 00000100:00080000:3.0:1789763130.842855:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763130.842863:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763130.842902:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565842816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763130.842933:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763130.842937:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763130.842938:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763130.842962:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3065 00000100:00080000:3.0:1789763130.842966:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763130.842971:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763130.842992:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763130.843006:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565842944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763130.843020:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565842816:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763130.843021:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763130.843032:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763130.843037:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763130.843043:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3065 00000100:00080000:3.0:1789763130.843046:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763130.843049:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763130.843054:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763130.843066:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565843072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763130.843077:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763130.843081:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763130.843083:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763130.843086:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3065 00000100:00080000:3.0:1789763130.843090:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763130.843093:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763130.843098:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1789763130.843109:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763130.843111:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565843200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763130.843172:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763130/real 1789763130] req@ffff93f844d67800 x1876699565842816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763130.843211:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565842816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763130.843227:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763130.843232:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763130.843238:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565842816:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763130.843267:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565842944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763130.843297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763130.843317:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763130/real 1789763130] req@ffff93f844d67100 x1876699565842944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763130.843340:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565842944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763130.843371:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763130.843377:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763130.843383:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565842944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763130.843394:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565843072:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763130.843441:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763130.843454:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763130/real 1789763130] req@ffff93f844d64e00 x1876699565843072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763130.843473:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565843072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763130.843501:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763130.843505:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763130.843509:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565843072:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763130.843516:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565843200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763130.843539:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763130.843550:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763130/real 1789763130] req@ffff93f844d66a00 x1876699565843200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763130.843563:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565843200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763130.843589:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763130.843592:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763130.843595:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565843200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763130.906867:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763131.930910:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763132.954920:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763133.978909:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763135.002875:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763135.962741:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763135.962748:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763135.962751:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763135.962759:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3070 00000100:00080000:3.0:1789763135.962769:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763135.962775:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763135.962806:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565843328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763135.962832:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763135.962835:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763135.962836:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763135.962839:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3070 00000100:00080000:3.0:1789763135.962841:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763135.962844:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763135.962852:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763135.962864:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565843456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763135.962865:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565843328:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763135.962879:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763135.962882:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763135.962883:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763135.962886:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3070 00000100:00080000:3.0:1789763135.962888:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763135.962890:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763135.962894:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763135.962920:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565843584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763135.962927:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763135.962929:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763135.962930:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763135.962933:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3070 00000100:00080000:3.0:1789763135.962937:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1789763135.962940:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763135.962943:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763135.962949:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763135.962958:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565843712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763135.962980:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763135/real 1789763135] req@ffff93f844d67480 x1876699565843328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763135.962994:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565843328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763135.963002:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763135.963005:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763135.963008:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565843328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763135.963026:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565843456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763135.963040:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763135.963048:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763135/real 1789763135] req@ffff93f844d65c00 x1876699565843456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763135.963056:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565843456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763135.963062:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763135.963064:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763135.963066:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565843456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763135.963070:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565843584:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763135.963080:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763135.963085:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763135/real 1789763135] req@ffff93f844d66d80 x1876699565843584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763135.963093:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565843584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763135.963098:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763135.963100:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763135.963101:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565843584:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763135.963105:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565843712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763135.963115:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763135.963121:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763135/real 1789763135] req@ffff93f844d65500 x1876699565843712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763135.963128:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565843712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763135.963133:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763135.963135:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763135.963136:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565843712:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763136.026819:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763137.050892:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763138.074976:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763139.098925:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763140.122861:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763141.082846:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763141.082859:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763141.082887:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763141.082902:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3076 00000100:00080000:0.0:1789763141.082916:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763141.082927:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763141.082979:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565843840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763141.083024:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763141.083030:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763141.083033:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763141.083039:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3076 00000100:00080000:0.0:1789763141.083044:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763141.083051:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763141.083066:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763141.083092:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565843968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763141.083114:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763141.083126:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:2.0:1789763141.083127:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565843840:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763141.083132:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763141.083154:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3076 00000100:00080000:0.0:1789763141.083159:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763141.083164:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763141.083171:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763141.083187:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565844096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763141.083206:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763141.083212:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763141.083217:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763141.083225:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3076 00000100:00080000:0.0:1789763141.083235:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763141.083246:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763141.083257:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789763141.083270:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763141.083287:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565844224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763141.083353:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763141/real 1789763141] req@ffff93f850d40000 x1876699565843840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763141.083386:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565843840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763141.083404:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763141.083410:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763141.083416:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565843840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763141.083462:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565843968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763141.083521:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763141.083563:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763141/real 1789763141] req@ffff93f850d40e00 x1876699565843968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763141.083583:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565843968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763141.083597:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763141.083605:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763141.083617:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565843968:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763141.083639:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565844096:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763141.083759:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763141.083804:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763141/real 1789763141] req@ffff93f850d42680 x1876699565844096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763141.083823:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565844096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763141.083835:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763141.083840:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763141.083844:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565844096:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763141.083852:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565844224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763141.083917:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763141.083933:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763141/real 1789763141] req@ffff93f850d41c00 x1876699565844224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763141.083949:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565844224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763141.083960:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763141.083964:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763141.083968:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565844224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789763141.146881:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763142.170955:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763143.194924:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763144.218904:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763145.242935:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763146.202895:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763146.202909:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763146.202916:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763146.202933:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3081 00000100:00080000:2.0:1789763146.202949:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763146.202960:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763146.203017:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565844352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763146.203058:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763146.203062:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763146.203065:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763146.203069:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3081 00000100:00080000:2.0:1789763146.203072:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763146.203076:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763146.203091:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763146.203106:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565844480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763146.203119:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763146.203124:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763146.203127:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763146.203137:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3081 00000100:00100000:1.0:1789763146.203138:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565844352:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763146.203144:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763146.203162:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763146.203170:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763146.203185:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565844608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763146.203196:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763146.203200:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763146.203202:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763146.203208:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3081 00000100:00080000:2.0:1789763146.203216:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763146.203223:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763146.203228:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763146.203239:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565844736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763146.203255:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763146.203317:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763146/real 1789763146] req@ffff93f8517c1880 x1876699565844352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763146.203352:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565844352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763146.203374:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763146.203382:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763146.203391:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565844352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763146.203434:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565844480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763146.203492:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763146.203524:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763146/real 1789763146] req@ffff93f8517c3480 x1876699565844480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763146.203541:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565844480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763146.203552:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763146.203557:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763146.203561:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565844480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763146.203569:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565844608:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763146.203593:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763146.203606:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763146/real 1789763146] req@ffff93f8517c3800 x1876699565844608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763146.203620:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565844608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763146.203629:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763146.203633:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763146.203637:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565844608:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763146.203689:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565844736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763146.203715:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763146.203729:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763146/real 1789763146] req@ffff93f8517c0e00 x1876699565844736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763146.203752:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565844736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763146.203762:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763146.203766:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763146.203770:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565844736:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763146.266940:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763147.290912:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763148.314932:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763149.338885:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763150.362899:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763151.322899:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763151.322916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763151.322923:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763151.322938:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3086 00000100:00080000:1.0:1789763151.322955:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763151.322968:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763151.323022:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565844864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763151.323076:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763151.323083:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763151.323087:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763151.323095:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3086 00000100:00080000:1.0:1789763151.323104:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763151.323113:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763151.323131:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763151.323154:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565844864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763151.323168:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565844992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763151.323199:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763151.323206:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763151.323210:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763151.323219:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3086 00000100:00080000:1.0:1789763151.323226:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763151.323234:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763151.323265:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789763151.323283:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763151.323297:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565845120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763151.323325:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763151.323332:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763151.323336:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763151.323345:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3086 00000100:00080000:1.0:1789763151.323353:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763151.323363:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763151.323374:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763151/real 1789763151] req@ffff93f844cca300 x1876699565844864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1789763151.323376:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763151.323421:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565845248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763151.323423:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565844864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763151.323443:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763151.323457:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763151.323473:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565844864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763151.323527:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565844992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763151.323575:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763151.323599:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763151/real 1789763151] req@ffff93f844cc8380 x1876699565844992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763151.323616:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565844992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763151.323627:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763151.323632:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763151.323637:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565844992:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763151.323677:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565845120:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763151.323704:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763151.323719:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763151/real 1789763151] req@ffff93f844cc8000 x1876699565845120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763151.323734:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565845120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763151.323746:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763151.323750:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763151.323754:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565845120:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763151.323761:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565845248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763151.323783:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763151.323817:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763151/real 1789763151] req@ffff93f844ccbb80 x1876699565845248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763151.323833:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565845248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763151.323844:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763151.323848:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763151.323852:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565845248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763151.386914:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763152.410937:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763153.434960:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763154.458862:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763155.482834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763156.442915:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763156.442927:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763156.442933:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763156.442945:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3091 00000100:00080000:3.0:1789763156.442958:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763156.442967:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763156.443035:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565845376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763156.443071:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763156.443075:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763156.443078:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763156.443083:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3091 00000100:00080000:3.0:1789763156.443087:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763156.443092:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763156.443105:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763156.443127:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565845376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763156.443128:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565845504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763156.443155:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763156.443159:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763156.443161:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763156.443167:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3091 00000100:00080000:3.0:1789763156.443171:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763156.443174:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763156.443180:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763156.443197:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565845632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763156.443210:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763156.443214:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763156.443215:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763156.443224:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3091 00000100:00080000:3.0:1789763156.443229:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789763156.443236:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763156.443242:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763156.443252:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763156.443268:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565845760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763156.443332:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763156/real 1789763156] req@ffff93f844d64380 x1876699565845376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763156.443361:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565845376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763156.443376:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763156.443381:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763156.443399:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565845376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763156.443442:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565845504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763156.443469:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763156.443491:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763156/real 1789763156] req@ffff93f844d64000 x1876699565845504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763156.443518:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565845504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763156.443533:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763156.443539:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763156.443546:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565845504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763156.443558:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565845632:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763156.443613:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763156.443630:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763156/real 1789763156] req@ffff93f844d64a80 x1876699565845632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763156.443654:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565845632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763156.443717:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763156.443722:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763156.443726:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565845632:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763156.443734:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565845760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763156.443805:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763156.443822:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763156/real 1789763156] req@ffff93f844d65f80 x1876699565845760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763156.443836:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565845760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763156.443865:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763156.443869:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763156.443873:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565845760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789763156.506889:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763157.530881:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763158.554771:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763159.578901:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763160.602871:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763161.563514:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763161.563532:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763161.563539:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763161.563573:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3096 00000100:00080000:0.0:1789763161.563588:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763161.563600:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763161.563737:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565845888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763161.563799:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763161.563808:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763161.563811:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763161.563821:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3096 00000100:00080000:0.0:1789763161.563830:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763161.563840:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763161.563857:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763161.563885:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565846016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763161.563906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763161.563912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763161.563916:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763161.563924:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3096 00000100:00080000:0.0:1789763161.563932:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763161.563940:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763161.563949:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763161.563970:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565846144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763161.563988:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763161.563992:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763161.563995:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763161.564004:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3096 00000100:00080000:0.0:1789763161.564011:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789763161.564013:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565845888:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763161.564019:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763161.564038:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763161.564069:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565846272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763161.564127:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763161.564198:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763161/real 1789763161] req@ffff93f850d41180 x1876699565845888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763161.564225:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565845888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763161.564250:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763161.564256:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763161.564262:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565845888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763161.564272:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565846016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763161.564302:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763161.564319:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763161/real 1789763161] req@ffff93f850d43100 x1876699565846016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763161.564335:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565846016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763161.564346:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763161.564350:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763161.564355:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565846016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763161.564385:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565846144:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763161.564408:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763161.564421:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763161/real 1789763161] req@ffff93f850d41f80 x1876699565846144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763161.564436:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565846144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763161.564447:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763161.564451:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763161.564455:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565846144:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763161.564463:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565846272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763161.564483:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763161.564494:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763161/real 1789763161] req@ffff93f850d41500 x1876699565846272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763161.564509:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565846272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763161.564519:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763161.564522:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763161.564526:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565846272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763161.626948:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763162.650926:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763163.674869:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763164.698895:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763165.722886:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763166.682905:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763166.682918:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763166.682923:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763166.682935:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3101 00000100:00080000:2.0:1789763166.682948:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763166.682957:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763166.683006:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565846400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763166.683059:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763166.683064:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763166.683067:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763166.683071:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3101 00000100:00080000:2.0:1789763166.683076:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763166.683081:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763166.683094:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763166.683119:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565846528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763166.683142:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1789763166.683144:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565846400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763166.683158:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763166.683165:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763166.683176:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3101 00000100:00080000:2.0:1789763166.683185:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763166.683194:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763166.683207:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763166.683231:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565846656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763166.683247:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763166.683251:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763166.683259:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1789763166.683261:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763166.683271:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3101 00000100:00080000:2.0:1789763166.683279:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763166.683286:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763166.683293:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763166.683309:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565846784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763166.683358:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763166/real 1789763166] req@ffff93f8517c0e00 x1876699565846400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763166.683390:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565846400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763166.683405:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763166.683411:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763166.683419:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565846400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763166.683453:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565846528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763166.683487:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763166.683511:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763166/real 1789763166] req@ffff93f8517c3800 x1876699565846528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763166.683526:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565846528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763166.683537:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763166.683541:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763166.683545:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565846528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763166.683552:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565846656:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763166.683572:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763166.683583:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763166/real 1789763166] req@ffff93f8517c3480 x1876699565846656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763166.683598:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565846656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763166.683608:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763166.683611:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763166.683615:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565846656:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763166.683621:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565846784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763166.683640:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763166.683733:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763166/real 1789763166] req@ffff93f8517c1880 x1876699565846784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763166.683758:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565846784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763166.683774:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763166.683781:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763166.683788:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565846784:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763166.746901:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763167.770791:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763168.794892:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763169.818851:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763170.842859:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763171.802736:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763171.802748:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763171.802753:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763171.802762:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3106 00000100:00080000:3.0:1789763171.802770:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763171.802776:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763171.802809:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565846912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763171.802836:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763171.802840:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763171.802843:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763171.802848:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3106 00000100:00080000:3.0:1789763171.802852:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763171.802857:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763171.802867:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763171.802871:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565846912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763171.802880:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565847040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763171.802890:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763171.802894:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763171.802896:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763171.802899:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3106 00000100:00080000:3.0:1789763171.802903:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763171.802906:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763171.802911:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763171.802921:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565847168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763171.802931:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1789763171.802932:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763171.802935:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763171.802937:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763171.802940:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3106 00000100:00080000:3.0:1789763171.802944:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763171.802947:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763171.802952:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763171.802962:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565847296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763171.802981:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763171/real 1789763171] req@ffff93f844d66300 x1876699565846912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763171.803000:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565846912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763171.803013:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763171.803018:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763171.803023:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565846912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763171.803045:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565847040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763171.803095:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763171.803114:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763171/real 1789763171] req@ffff93f844d65180 x1876699565847040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763171.803130:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565847040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763171.803141:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763171.803145:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763171.803149:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565847040:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763171.803156:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565847168:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763171.803176:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763171.803188:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763171/real 1789763171] req@ffff93f844d65f80 x1876699565847168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763171.803211:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565847168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763171.803221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763171.803224:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763171.803228:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565847168:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763171.803234:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565847296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763171.803265:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763171.803271:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763171/real 1789763171] req@ffff93f844d64a80 x1876699565847296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763171.803278:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565847296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763171.803283:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763171.803285:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763171.803287:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565847296:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789763171.866804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763172.890875:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763173.914882:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763174.938871:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763175.962868:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763176.922872:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763176.922883:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763176.922889:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763176.922903:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3111 00000100:00080000:3.0:1789763176.922916:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763176.922927:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763176.922972:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565847424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763176.923011:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763176.923017:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763176.923020:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763176.923026:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3111 00000100:00080000:3.0:1789763176.923031:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763176.923037:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763176.923049:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763176.923075:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565847552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763176.923081:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565847424:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763176.923109:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763176.923114:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763176.923117:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763176.923123:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3111 00000100:00080000:3.0:1789763176.923131:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763176.923138:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763176.923146:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763176.923171:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565847680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763176.923188:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763176.923194:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763176.923197:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763176.923202:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3111 00000100:00080000:3.0:1789763176.923212:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789763176.923219:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763176.923226:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763176.923249:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763176.923269:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565847808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763176.923293:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763176/real 1789763176] req@ffff93f844d64000 x1876699565847424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763176.923333:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565847424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763176.923370:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763176.923379:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763176.923388:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565847424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763176.923434:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565847552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763176.923477:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763176.923501:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763176/real 1789763176] req@ffff93f844d64380 x1876699565847552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763176.923525:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565847552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763176.923538:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763176.923543:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763176.923547:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565847552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763176.923558:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565847680:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763176.923585:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763176.923599:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763176/real 1789763176] req@ffff93f844d65500 x1876699565847680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763176.923706:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565847680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763176.923727:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763176.923736:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763176.923745:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565847680:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763176.923764:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565847808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763176.923825:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763176.923861:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763176/real 1789763176] req@ffff93f844d66d80 x1876699565847808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763176.923890:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565847808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763176.923909:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763176.923916:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763176.923923:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565847808:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763176.986854:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763178.010917:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763179.034903:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763180.058871:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763181.082852:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763182.042829:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763182.042864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763182.042872:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763182.042887:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3117 00000100:00080000:2.0:1789763182.042903:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763182.042916:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763182.042973:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565847936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763182.043012:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763182.043018:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763182.043020:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763182.043026:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3117 00000100:00080000:2.0:1789763182.043033:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763182.043042:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763182.043066:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763182.043075:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565847936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763182.043091:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565848064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763182.043108:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763182.043113:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763182.043116:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763182.043120:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3117 00000100:00080000:2.0:1789763182.043125:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763182.043130:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763182.043136:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763182.043152:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565848192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763182.043179:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763182.043186:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763182.043190:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763182.043199:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3117 00000100:00080000:2.0:1789763182.043208:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763182.043218:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763182.043228:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763182.043243:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565848064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763182.043260:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565848320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763182.043308:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565848192:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763182.043348:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565848320:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763182.106826:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763183.130855:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763184.154881:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789763185.114979:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789763185.115037:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789763185.115063:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789763185.115119:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789763185.115139:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789763185.115153:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1789763185.115193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763182/real 1789763185] req@ffff93f8517c2d80 x1876699565848320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763185.115238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565848320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763185.115259:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763185.115267:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763185.115276:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565848320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763185.115291:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763182/real 1789763185] req@ffff93f8517c3100 x1876699565848064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763185.115316:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565848064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763185.115330:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763185.115336:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763185.115342:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565848064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763185.115354:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763182/real 1789763185] req@ffff93f8517c1180 x1876699565848192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763185.115375:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565848192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763185.115391:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763185.115398:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763185.115404:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565848192:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763185.115417:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763182/real 1789763185] req@ffff93f8517c2a00 x1876699565847936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763185.115441:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565847936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763185.115456:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763185.115461:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763185.115467:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565847936:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789763185.178828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763186.202842:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763187.162878:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763187.162891:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763187.162896:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763187.162909:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3122 00000100:00080000:0.0:1789763187.162920:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763187.162929:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763187.162982:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565848448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763187.163032:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763187.163039:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763187.163043:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763187.163052:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3122 00000100:00080000:0.0:1789763187.163060:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763187.163069:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763187.163086:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763187.163115:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565848576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763187.163132:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565848448:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763187.163169:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763187.163177:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763187.163182:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763187.163191:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3122 00000100:00080000:0.0:1789763187.163201:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763187.163210:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763187.163224:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789763187.163253:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763187.163258:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565848704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763187.163292:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763187.163298:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763187.163302:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763187.163311:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3122 00000100:00080000:0.0:1789763187.163320:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1789763187.163330:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763187/real 1789763187] req@ffff93f850d43480 x1876699565848448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763187.163333:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763187.163349:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763187.163378:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565848832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763187.163393:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565848448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763187.163424:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763187.163433:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763187.163442:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565848448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763187.163485:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565848576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763187.163575:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763187.163604:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763187/real 1789763187] req@ffff93f850d43800 x1876699565848576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763187.163632:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565848576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763187.163694:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763187.163702:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763187.163708:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565848576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763187.163722:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565848704:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763187.163766:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763187.163789:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763187/real 1789763187] req@ffff93f850d41500 x1876699565848704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763187.163812:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565848704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763187.163827:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763187.163833:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763187.163839:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565848704:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763187.164638:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565848832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763187.164732:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763187.164760:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763187/real 1789763187] req@ffff93f850d41f80 x1876699565848832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763187.164801:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565848832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763187.164820:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763187.164827:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763187.164835:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565848832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789763187.226862:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763188.250888:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763189.274882:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763190.298888:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763191.322848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763192.282817:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763192.282831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763192.282838:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763192.282857:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3127 00000100:00080000:1.0:1789763192.282874:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763192.282888:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763192.282948:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565848960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763192.283010:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763192.283018:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763192.283023:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763192.283033:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3127 00000100:00080000:1.0:1789763192.283044:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763192.283054:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763192.283065:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565848960:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789763192.283071:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763192.283099:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565849088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763192.283125:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763192.283142:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763192.283145:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763192.283151:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3127 00000100:00080000:1.0:1789763192.283159:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763192.283164:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763192.283172:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789763192.283180:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763192.283197:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565849216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763192.283215:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763192.283219:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763192.283221:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763192.283226:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3127 00000100:00080000:1.0:1789763192.283231:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763192.283237:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763192.283244:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763192.283259:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565849344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763192.283267:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763192/real 1789763192] req@ffff93f844cc8700 x1876699565848960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763192.283312:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565848960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763192.283328:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763192.283334:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763192.283339:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565848960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763192.283371:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565849088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763192.283405:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763192.283422:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763192/real 1789763192] req@ffff93f844ccb800 x1876699565849088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763192.283438:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565849088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763192.283449:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763192.283454:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763192.283458:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565849088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763192.283465:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565849216:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763192.283486:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763192.283539:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763192/real 1789763192] req@ffff93f844ccb480 x1876699565849216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763192.283555:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565849216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763192.283565:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763192.283569:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763192.283573:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565849216:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763192.283580:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565849344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763192.283607:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763192.283620:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763192/real 1789763192] req@ffff93f844ccaa00 x1876699565849344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763192.283634:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565849344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763192.283679:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763192.283683:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763192.283689:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565849344:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763192.346860:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763193.370870:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763194.394881:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763195.418866:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763196.442823:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763197.402847:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763197.402862:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763197.402869:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763197.402883:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3132 00000100:00080000:1.0:1789763197.402897:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763197.402907:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763197.402956:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565849472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763197.402998:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763197.403003:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763197.403005:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763197.403010:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3132 00000100:00080000:1.0:1789763197.403014:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763197.403020:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763197.403033:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763197.403052:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565849600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763197.403082:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565849472:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763197.403083:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763197.403108:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763197.403113:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763197.403124:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3132 00000100:00080000:1.0:1789763197.403139:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763197.403151:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763197.403162:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763197.403185:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565849728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763197.403211:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763197.403224:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:3.0:1789763197.403226:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763197.403234:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763197.403247:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3132 00000100:00080000:1.0:1789763197.403259:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763197.403268:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763197.403279:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763197.403307:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565849856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763197.403324:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763197/real 1789763197] req@ffff93f844cc9c00 x1876699565849472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763197.403386:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565849472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763197.403408:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763197.403418:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763197.403425:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565849472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763197.403493:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565849600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763197.403527:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763197.403545:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763197/real 1789763197] req@ffff93f844cca680 x1876699565849600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763197.403561:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565849600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763197.403573:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763197.403577:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763197.403581:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565849600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763197.403589:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565849728:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763197.403616:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763197.403628:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763197/real 1789763197] req@ffff93f844cc8e00 x1876699565849728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763197.403679:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565849728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763197.403692:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763197.403697:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763197.403701:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565849728:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763197.403709:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565849856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763197.403736:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763197.403750:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763197/real 1789763197] req@ffff93f844ccb100 x1876699565849856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763197.403765:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565849856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763197.403775:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763197.403780:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763197.403783:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565849856:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763197.466863:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763198.490853:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763199.514917:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763200.538763:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763201.562816:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763202.522777:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763202.522790:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763202.522807:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763202.522821:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3137 00000100:00080000:1.0:1789763202.522834:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763202.522844:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763202.522891:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565849984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763202.522928:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763202.522933:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763202.522936:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763202.522941:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3137 00000100:00080000:1.0:1789763202.522946:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763202.522951:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763202.522962:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763202.522977:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565850112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763202.523003:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763202.523008:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763202.523010:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763202.523018:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3137 00000100:00080000:1.0:1789763202.523025:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763202.523033:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763202.523043:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763202.523066:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565849984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763202.523070:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565850240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763202.523100:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763202.523107:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763202.523112:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763202.523121:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3137 00000100:00080000:1.0:1789763202.523132:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763202.523138:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763202.523146:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789763202.523164:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763202.523170:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565850368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763202.523248:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763202/real 1789763202] req@ffff93f844ccad80 x1876699565849984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763202.523276:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565849984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763202.523293:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763202.523309:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763202.523315:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565849984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763202.523324:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565850112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763202.523351:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763202.523366:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763202/real 1789763202] req@ffff93f844cc9500 x1876699565850112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763202.523381:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565850112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763202.523392:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763202.523395:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763202.523400:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565850112:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763202.523462:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565850240:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763202.523489:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763202.523503:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763202/real 1789763202] req@ffff93f844ccb100 x1876699565850240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763202.523518:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565850240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763202.523537:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763202.523541:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763202.523545:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565850240:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763202.523551:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565850368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763202.523571:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763202.523582:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763202/real 1789763202] req@ffff93f844cc8e00 x1876699565850368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763202.523596:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565850368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763202.523605:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763202.523609:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763202.523613:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565850368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1789763202.586898:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763203.610920:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763204.634878:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763205.658844:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763206.682872:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763207.642798:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763207.642814:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763207.642828:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763207.642842:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3142 00000100:00080000:0.0:1789763207.642856:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763207.642866:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763207.642914:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565850496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763207.642946:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763207.642951:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763207.642953:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763207.642957:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3142 00000100:00080000:0.0:1789763207.642961:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763207.642965:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763207.642975:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763207.642990:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565850624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763207.643003:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763207.643009:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763207.643011:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1789763207.643011:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565850496:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763207.643021:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3142 00000100:00080000:0.0:1789763207.643028:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763207.643031:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763207.643053:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763207.643075:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565850752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763207.643090:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763207.643095:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763207.643097:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763207.643101:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3142 00000100:00080000:0.0:1789763207.643106:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763207.643110:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763207.643116:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763207.643128:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565850880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763207.643165:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763207.643250:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763207/real 1789763207] req@ffff93f850d43100 x1876699565850496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763207.643292:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565850496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763207.643316:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763207.643325:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763207.643334:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565850496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763207.643406:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565850624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763207.643445:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763207.643468:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763207/real 1789763207] req@ffff93f850d41180 x1876699565850624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763207.643491:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565850624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763207.643507:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763207.643513:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763207.643518:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565850624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763207.643529:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565850752:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763207.643561:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763207.643580:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763207/real 1789763207] req@ffff93f850d41c00 x1876699565850752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763207.643599:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565850752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763207.643613:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763207.643619:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763207.643624:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565850752:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763207.643633:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565850880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763207.643716:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763207.643735:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763207/real 1789763207] req@ffff93f850d42680 x1876699565850880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763207.643755:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565850880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763207.643768:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763207.643773:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763207.643778:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565850880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763207.706839:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763208.730920:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763209.754841:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763210.778867:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789763211.802821:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763212.762876:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763212.762893:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763212.762900:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763212.762917:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3147 00000100:00080000:0.0:1789763212.762933:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763212.762947:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763212.763009:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565851008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763212.763053:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763212.763059:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763212.763064:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763212.763075:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3147 00000100:00080000:0.0:1789763212.763085:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763212.763092:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763212.763108:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763212.763111:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565851008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763212.763147:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565851136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763212.763175:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763212.763183:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763212.763190:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1789763212.763196:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763212.763213:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3147 00000100:00080000:0.0:1789763212.763228:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763212.763239:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763212.763250:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763212.763251:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763212/real 1789763212] req@ffff93f850d40e00 x1876699565851008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763212.763276:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565851264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763212.763281:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565851008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763212.763363:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763212.763368:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763212.763372:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:0.0:1789763212.763388:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763212.763392:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789763212.763400:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565851008:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763212.763403:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3147 00000100:00080000:0.0:1789763212.763424:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763212.763433:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763212.763454:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763212.763477:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565851136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763212.763478:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565851392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763212.763565:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763212.763607:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763212/real 1789763212] req@ffff93f850d40000 x1876699565851136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763212.763631:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565851136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763212.763707:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763212.763714:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763212.763719:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565851136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763212.763738:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565851264:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763212.763808:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763212.763857:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763212/real 1789763212] req@ffff93f850d42300 x1876699565851264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763212.763875:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565851264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763212.763885:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763212.763889:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763212.763893:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565851264:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763212.763912:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565851392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763212.763939:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763212.763955:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763212/real 1789763212] req@ffff93f850d40380 x1876699565851392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763212.763970:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565851392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763212.763981:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763212.763984:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763212.763987:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565851392:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763212.826863:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763213.850832:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763214.874859:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763215.898897:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763216.922934:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763217.882857:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763217.882873:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763217.882880:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763217.882897:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3152 00000100:00080000:3.0:1789763217.882914:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763217.882928:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763217.882986:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565851520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763217.883024:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763217.883032:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763217.883036:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763217.883045:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3152 00000100:00080000:3.0:1789763217.883054:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763217.883063:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763217.883079:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763217.883095:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565851520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763217.883104:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565851648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763217.883124:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763217.883130:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763217.883133:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763217.883140:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3152 00000100:00080000:3.0:1789763217.883146:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763217.883152:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763217.883160:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763217.883177:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565851776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763217.883186:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763217.883195:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763217.883201:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763217.883204:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763217.883210:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3152 00000100:00080000:3.0:1789763217.883216:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763217.883242:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763217.883253:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763217.883257:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763217/real 1789763217] req@ffff93f844d65c00 x1876699565851520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763217.883274:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565851904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763217.883293:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565851520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763217.883336:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763217.883344:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763217.883352:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565851520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763217.883393:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565851648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763217.883440:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763217.883461:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763217/real 1789763217] req@ffff93f844d67480 x1876699565851648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763217.883477:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565851648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763217.883488:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763217.883493:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763217.883497:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565851648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763217.883504:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565851776:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763217.883528:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763217.883539:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763217/real 1789763217] req@ffff93f844d66a00 x1876699565851776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763217.883562:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565851776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763217.883572:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763217.883575:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763217.883579:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565851776:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763217.883585:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565851904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763217.883606:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763217.883617:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763217/real 1789763217] req@ffff93f844d64e00 x1876699565851904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763217.883631:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565851904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763217.883640:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763217.883676:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763217.883680:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565851904:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763217.946898:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763218.970876:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763219.994920:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763221.018883:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763222.042863:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763223.002855:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763223.002867:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763223.002872:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763223.002884:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3157 00000100:00080000:0.0:1789763223.002896:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763223.002906:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763223.002951:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565852032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763223.003000:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763223.003005:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763223.003007:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763223.003012:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3157 00000100:00080000:0.0:1789763223.003016:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763223.003022:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763223.003034:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763223.003052:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565852160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763223.003068:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565852032:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763223.003084:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763223.003094:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763223.003097:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763223.003107:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3157 00000100:00080000:0.0:1789763223.003117:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763223.003127:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763223.003138:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:1.0:1789763223.003160:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763223.003171:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565852288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763223.003196:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763223.003200:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763223.003203:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763223.003211:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3157 00000100:00080000:0.0:1789763223.003219:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1789763223.003228:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763223/real 1789763223] req@ffff93f850d40700 x1876699565852032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763223.003231:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763223.003245:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763223.003301:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565852416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763223.003314:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565852032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763223.003336:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763223.003342:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763223.003350:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565852032:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763223.003382:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565852160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763223.003417:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763223.003435:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763223/real 1789763223] req@ffff93f850d43b80 x1876699565852160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763223.003451:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565852160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763223.003462:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763223.003466:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763223.003471:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565852160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763223.003478:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565852288:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763223.003500:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763223.003511:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763223/real 1789763223] req@ffff93f850d40380 x1876699565852288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763223.003525:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565852288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763223.003536:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763223.003539:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763223.003543:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565852288:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763223.003550:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565852416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763223.003569:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763223.003580:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763223/real 1789763223] req@ffff93f850d42300 x1876699565852416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763223.003602:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565852416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763223.003618:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763223.003625:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763223.003631:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565852416:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763223.066855:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763224.090820:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763225.114871:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763226.138859:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763227.162890:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763228.122734:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763228.122746:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763228.122752:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763228.122764:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3163 00000100:00080000:3.0:1789763228.122774:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763228.122783:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763228.122822:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565852544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763228.122851:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763228.122856:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763228.122857:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763228.122862:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3163 00000100:00080000:3.0:1789763228.122866:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763228.122871:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763228.122882:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763228.122897:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565852672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763228.122908:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763228.122912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763228.122914:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763228.122918:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3163 00000100:00080000:3.0:1789763228.122922:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763228.122926:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763228.122931:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763228.122944:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565852800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763228.122955:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763228.122959:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763228.122961:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763228.122965:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3163 00000100:00080000:3.0:1789763228.122968:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763228.122972:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763228.122977:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763228.122988:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565852928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763228.123151:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565852544:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1789763228.123302:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763228.123378:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763228/real 1789763228] req@ffff93f844d67100 x1876699565852544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763228.123410:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565852544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763228.123454:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763228.123458:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763228.123462:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565852544:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763228.123471:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565852672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763228.123499:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763228.123514:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763228/real 1789763228] req@ffff93f844d67800 x1876699565852672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763228.123527:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565852672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763228.123536:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763228.123540:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763228.123544:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565852672:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763228.123548:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565852800:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763228.123566:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763228.123574:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763228/real 1789763228] req@ffff93f844d64700 x1876699565852800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763228.123582:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565852800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763228.123588:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763228.123590:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763228.123592:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565852800:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763228.123595:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565852928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763228.123606:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763228.123612:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763228/real 1789763228] req@ffff93f844d67b80 x1876699565852928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763228.123624:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565852928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763228.123629:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763228.123630:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763228.123632:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565852928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763228.186862:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763229.210880:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763230.234883:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763231.258953:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763232.282765:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763233.242786:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763233.242798:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763233.242803:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763233.242816:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3168 00000100:00080000:3.0:1789763233.242833:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763233.242845:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763233.242911:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565853056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763233.242932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763233.242935:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763233.242936:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763233.242939:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3168 00000100:00080000:3.0:1789763233.242962:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763233.242968:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763233.242981:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763233.242993:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565853056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763233.243007:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565853184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763233.243025:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763233.243030:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763233.243033:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763233.243064:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3168 00000100:00080000:3.0:1789763233.243068:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763233.243071:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763233.243076:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:1.0:1789763233.243089:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763233.243091:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565853312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763233.243107:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763233.243111:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763233.243112:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763233.243115:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3168 00000100:00080000:3.0:1789763233.243119:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763233.243122:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763233.243126:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763233.243136:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565853440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763233.243142:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763233/real 1789763233] req@ffff93f844d66680 x1876699565853056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763233.243166:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565853056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763233.243176:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763233.243180:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763233.243184:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565853056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763233.243245:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565853184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763233.243278:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763233.243295:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763233/real 1789763233] req@ffff93f844d65880 x1876699565853184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763233.243311:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565853184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763233.243322:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763233.243327:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763233.243331:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565853184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763233.243338:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565853312:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763233.243359:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763233.243369:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763233/real 1789763233] req@ffff93f844d67b80 x1876699565853312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763233.243383:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565853312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763233.243393:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763233.243396:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763233.243400:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565853312:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763233.243406:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565853440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763233.243450:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763233.243456:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763233/real 1789763233] req@ffff93f844d64700 x1876699565853440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763233.243463:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565853440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763233.243468:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763233.243470:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763233.243472:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565853440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763233.306743:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763234.330762:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763235.354833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763236.378855:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763237.402836:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763238.362827:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763238.362841:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763238.362846:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763238.362859:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3173 00000100:00080000:0.0:1789763238.362871:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763238.362880:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763238.362927:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565853568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763238.362970:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763238.362975:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763238.362977:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763238.362982:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3173 00000100:00080000:0.0:1789763238.362986:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763238.362992:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763238.363003:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763238.363019:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565853696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763238.363035:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565853568:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763238.363038:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763238.363053:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763238.363056:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763238.363062:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3173 00000100:00080000:0.0:1789763238.363066:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763238.363071:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763238.363078:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763238.363097:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565853824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763238.363109:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763238.363114:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763238.363118:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763238.363123:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3173 00000100:00080000:0.0:1789763238.363130:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1789763238.363141:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763238.363142:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763238.363182:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763238.363203:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565853952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763238.363229:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763238/real 1789763238] req@ffff93f850d40000 x1876699565853568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763238.363259:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565853568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763238.363273:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763238.363279:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763238.363285:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565853568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763238.363313:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565853696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763238.363358:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763238.363375:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763238/real 1789763238] req@ffff93f850d40e00 x1876699565853696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763238.363391:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565853696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763238.363402:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763238.363406:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763238.363411:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565853696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763238.363418:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565853824:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763238.363440:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763238.363451:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763238/real 1789763238] req@ffff93f850d42680 x1876699565853824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763238.363465:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565853824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763238.363475:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763238.363478:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763238.363482:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565853824:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763238.363488:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565853952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763238.363509:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763238.363520:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763238/real 1789763238] req@ffff93f850d41c00 x1876699565853952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763238.363533:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565853952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763238.363543:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763238.363547:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763238.363550:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565853952:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763238.426845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763239.450911:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763240.474881:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763241.498869:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763242.522915:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763243.482895:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763243.482912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763243.482921:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763243.482942:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3178 00000100:00080000:3.0:1789763243.482961:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763243.482974:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763243.483044:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565854080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763243.483086:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763243.483092:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763243.483094:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763243.483100:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3178 00000100:00080000:3.0:1789763243.483106:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763243.483137:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763243.483154:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763243.483180:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565854080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763243.483181:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565854208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763243.483233:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763243.483239:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763243.483243:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763243.483251:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3178 00000100:00080000:3.0:1789763243.483258:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763243.483262:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763243.483270:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763243.483289:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565854336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763243.483301:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763243.483333:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763243.483342:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763243.483347:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763243.483361:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3178 00000100:00080000:3.0:1789763243.483372:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763243.483384:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763243.483390:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763243/real 1789763243] req@ffff93f844d67800 x1876699565854080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1789763243.483404:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763243.483423:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565854080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763243.483436:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565854464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763243.483465:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763243.483497:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763243.483505:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565854080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763243.483548:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565854208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763243.483584:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763243.483601:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763243/real 1789763243] req@ffff93f844d67100 x1876699565854208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763243.483617:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565854208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763243.483628:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763243.483633:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763243.483637:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565854208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763243.483697:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565854336:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763243.483725:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763243.483740:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763243/real 1789763243] req@ffff93f844d64e00 x1876699565854336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763243.483755:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565854336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763243.483766:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763243.483783:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763243.483788:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565854336:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763243.483796:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565854464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763243.483820:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763243.483833:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763243/real 1789763243] req@ffff93f844d66a00 x1876699565854464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763243.483852:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565854464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763243.483869:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763243.483876:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763243.483882:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565854464:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763243.546843:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763244.570893:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763245.594929:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763246.618884:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763247.642905:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763248.602771:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763248.602787:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763248.602792:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763248.602808:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3183 00000100:00080000:2.0:1789763248.602818:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763248.602827:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763248.602870:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565854592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763248.602914:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763248.602920:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763248.602924:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763248.602930:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3183 00000100:00080000:2.0:1789763248.602936:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763248.602941:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763248.602951:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763248.602970:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565854720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763248.602982:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763248.602985:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763248.602987:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763248.602992:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3183 00000100:00080000:2.0:1789763248.602996:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763248.603002:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763248.603008:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763248.603022:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565854848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763248.603037:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763248.603041:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763248.603046:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789763248.603057:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565854592:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763248.603057:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3183 00000100:00080000:2.0:1789763248.603070:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763248.603098:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763248.603120:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763248.603137:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565854976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763248.603308:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565854720:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763248.603395:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565854848:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763248.603436:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565854976:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763248.666739:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763249.690834:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763250.714845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789763251.675126:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789763251.675189:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789763251.675216:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763251.675295:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763251.675314:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763251.675341:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789763251.675382:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763248/real 1789763251] req@ffff93f8517c0380 x1876699565854848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763251.675432:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565854848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763251.675454:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763251.675464:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763251.675472:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565854848:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763251.675487:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763248/real 1789763251] req@ffff93f8517c1500 x1876699565854976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763251.675511:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565854976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763251.675527:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763251.675536:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763251.675543:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565854976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763251.675556:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763248/real 1789763251] req@ffff93f8517c2680 x1876699565854592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763251.675598:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565854592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763251.675613:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763251.675618:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763251.675623:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565854592:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763251.675634:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763248/real 1789763251] req@ffff93f8517c0a80 x1876699565854720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763251.675754:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565854720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763251.675771:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763251.675777:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763251.675784:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565854720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763251.738845:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763252.762895:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763253.722803:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763253.722816:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763253.722820:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763253.722835:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3188 00000100:00080000:3.0:1789763253.722849:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763253.722859:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763253.722906:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565855104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763253.722945:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763253.722950:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763253.722953:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763253.722958:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3188 00000100:00080000:3.0:1789763253.722963:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763253.722969:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763253.722982:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763253.723004:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565855104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763253.723012:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565855232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763253.723060:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763253.723068:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763253.723072:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763253.723080:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3188 00000100:00080000:3.0:1789763253.723089:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763253.723098:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763253.723111:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789763253.723142:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763253.723153:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565855360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763253.723179:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763253.723184:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763253.723187:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763253.723193:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3188 00000100:00080000:3.0:1789763253.723199:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763253.723204:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763253.723212:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763253.723231:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763253/real 1789763253] req@ffff93f844d67480 x1876699565855104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763253.723239:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565855488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763253.723300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565855104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763253.723319:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763253.723325:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763253.723331:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565855104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763253.723364:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565855232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763253.723411:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763253.723450:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763253/real 1789763253] req@ffff93f844d65c00 x1876699565855232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763253.723479:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565855232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763253.723498:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763253.723505:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763253.723511:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565855232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763253.723525:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565855360:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763253.723569:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763253.723587:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763253/real 1789763253] req@ffff93f844d66d80 x1876699565855360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763253.723605:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565855360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763253.723624:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763253.723631:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763253.723637:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565855360:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763253.723695:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565855488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763253.723741:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763253.723767:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763253/real 1789763253] req@ffff93f844d65500 x1876699565855488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763253.723792:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565855488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763253.723809:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763253.723816:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763253.723823:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565855488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763253.786924:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763254.810904:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763255.834813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763256.858929:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763257.882842:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763258.842847:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763258.842869:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763258.842880:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763258.842901:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3193 00000100:00080000:3.0:1789763258.842918:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763258.842929:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763258.842992:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565855616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763258.843070:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763258.843079:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763258.843083:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763258.843094:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3193 00000100:00080000:3.0:1789763258.843105:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763258.843115:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763258.843136:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565855616:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789763258.843138:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763258.843175:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565855744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763258.843193:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763258.843199:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763258.843201:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763258.843207:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3193 00000100:00080000:3.0:1789763258.843214:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763258.843219:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763258.843226:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763258.843242:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565855872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763258.843257:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763258.843262:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763258.843264:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763258.843268:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3193 00000100:00080000:3.0:1789763258.843275:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789763258.843283:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763258.843289:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763258.843300:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763258.843318:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565856000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763258.843366:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763258/real 1789763258] req@ffff93f844d64380 x1876699565855616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763258.843395:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565855616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763258.843411:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763258.843416:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763258.843439:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565855616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763258.843487:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565855744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763258.843520:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763258.843537:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763258/real 1789763258] req@ffff93f844d64000 x1876699565855744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763258.843555:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565855744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763258.843567:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763258.843571:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763258.843577:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565855744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763258.843589:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565855872:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763258.843632:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763258.843701:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763258/real 1789763258] req@ffff93f844d64a80 x1876699565855872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763258.843720:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565855872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763258.843731:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763258.843736:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763258.843740:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565855872:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763258.843748:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565856000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763258.843779:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763258.843793:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763258/real 1789763258] req@ffff93f844d65f80 x1876699565856000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763258.843808:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565856000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763258.843827:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763258.843831:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763258.843835:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565856000:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763258.906828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763259.930794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763260.954912:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763261.978848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763263.002849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763263.962782:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763263.962795:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763263.962801:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763263.962815:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3198 00000100:00080000:3.0:1789763263.962829:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763263.962840:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763263.962889:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565856128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763263.962929:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763263.962936:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763263.962941:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763263.962949:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3198 00000100:00080000:3.0:1789763263.962956:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763263.962986:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763263.963002:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763263.963028:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565856256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763263.963031:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565856128:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763263.963049:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763263.963056:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763263.963059:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763263.963066:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3198 00000100:00080000:3.0:1789763263.963072:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763263.963079:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763263.963087:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763263.963106:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565856384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763263.963123:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763263.963124:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763263.963130:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763263.963133:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763263.963139:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3198 00000100:00080000:3.0:1789763263.963145:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763263.963150:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763263.963158:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763263.963176:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565856512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763263.963179:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763263/real 1789763263] req@ffff93f844d65180 x1876699565856128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763263.963210:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565856128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763263.963231:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763263.963239:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763263.963247:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565856128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763263.963290:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565856256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763263.963338:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763263.963364:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763263/real 1789763263] req@ffff93f844d66300 x1876699565856256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763263.963389:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565856256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763263.963400:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763263.963405:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763263.963409:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565856256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763263.963418:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565856384:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763263.963447:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763263.963461:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763263/real 1789763263] req@ffff93f844d65f80 x1876699565856384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763263.963476:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565856384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763263.963486:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763263.963489:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763263.963493:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565856384:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763263.963499:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565856512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763263.963518:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763263.963529:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763263/real 1789763263] req@ffff93f844d64a80 x1876699565856512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763263.963543:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565856512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763263.963553:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763263.963556:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763263.963560:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565856512:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763264.026852:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763265.050852:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763266.074840:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763267.098866:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763268.122901:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763269.082799:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763269.082813:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763269.082819:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763269.082832:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3204 00000100:00080000:1.0:1789763269.082845:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763269.082855:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763269.082898:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565856640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763269.082961:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763269.082968:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763269.082971:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763269.082976:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3204 00000100:00080000:1.0:1789763269.082982:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763269.082990:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763269.083002:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763269.083022:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565856768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763269.083036:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565856640:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763269.083042:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763269.083056:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763269.083060:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763269.083069:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3204 00000100:00080000:1.0:1789763269.083079:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763269.083086:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763269.083097:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763269.083124:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565856896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763269.083147:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1789763269.083159:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763269.083160:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763269.083170:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763269.083182:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3204 00000100:00080000:1.0:1789763269.083194:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763269.083204:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763269.083231:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763269/real 1789763269] req@ffff93f844cca680 x1876699565856640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1789763269.083233:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763269.083273:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565857024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763269.083274:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565856640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763269.083338:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763269.083347:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763269.083355:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565856640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763269.083388:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565856768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763269.083418:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763269.083433:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763269/real 1789763269] req@ffff93f844cc9c00 x1876699565856768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763269.083449:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565856768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763269.083460:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763269.083464:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763269.083468:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565856768:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763269.083475:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565856896:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763269.083494:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763269.083505:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763269/real 1789763269] req@ffff93f844ccaa00 x1876699565856896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763269.083520:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565856896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763269.083537:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763269.083542:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763269.083546:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565856896:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763269.083552:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565857024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763269.083582:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763269.083602:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763269/real 1789763269] req@ffff93f844ccb480 x1876699565857024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763269.083626:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565857024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763269.083681:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763269.083687:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763269.083692:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565857024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763269.146829:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763270.170828:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763271.194987:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763272.218895:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763273.242954:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763274.202919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763274.202934:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763274.202941:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763274.202959:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3209 00000100:00080000:2.0:1789763274.202980:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763274.202995:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763274.203073:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565857152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763274.203119:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763274.203126:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763274.203129:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763274.203135:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3209 00000100:00080000:2.0:1789763274.203142:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763274.203149:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763274.203165:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763274.203186:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565857152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763274.203188:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565857280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763274.203221:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763274.203228:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763274.203231:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763274.203238:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3209 00000100:00080000:2.0:1789763274.203243:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763274.203249:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763274.203257:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763274.203274:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565857408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763274.203290:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763274.203291:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763274.203299:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763274.203303:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763274.203310:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3209 00000100:00080000:2.0:1789763274.203318:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763274.203323:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763274.203332:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763274.203351:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763274/real 1789763274] req@ffff93f8517c0a80 x1876699565857152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763274.203353:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565857536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763274.203487:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565857152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763274.203505:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763274.203514:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763274.203523:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565857152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763274.203573:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565857280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763274.203622:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763274.203637:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763274/real 1789763274] req@ffff93f8517c2680 x1876699565857280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763274.203649:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565857280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763274.203683:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763274.203688:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763274.203692:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565857280:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763274.203699:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565857408:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763274.203720:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763274.203731:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763274/real 1789763274] req@ffff93f8517c1500 x1876699565857408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763274.203742:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565857408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763274.203750:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763274.203753:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763274.203756:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565857408:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763274.203762:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565857536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763274.203778:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763274.203787:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763274/real 1789763274] req@ffff93f8517c0380 x1876699565857536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763274.203800:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565857536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763274.203812:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763274.203817:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763274.203822:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565857536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763274.266966:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763275.290976:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763276.314972:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763277.338927:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763278.363083:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763279.322872:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763279.322881:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763279.322885:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763279.322893:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3214 00000100:00080000:1.0:1789763279.322905:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763279.322913:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763279.322949:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565857664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763279.322974:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763279.322977:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763279.322979:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763279.322983:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3214 00000100:00080000:1.0:1789763279.322986:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763279.322990:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763279.322998:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763279.323009:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565857792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763279.323017:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763279.323020:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763279.323023:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763279.323031:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3214 00000100:00100000:2.0:1789763279.323035:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565857664:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763279.323037:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763279.323053:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763279.323062:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763279.323070:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565857920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763279.323079:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763279.323081:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763279.323083:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763279.323088:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3214 00000100:00080000:1.0:1789763279.323090:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763279.323094:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763279.323098:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763279.323105:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565858048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763279.323162:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763279.323245:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763279/real 1789763279] req@ffff93f844ccb800 x1876699565857664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763279.323281:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565857664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763279.323304:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763279.323313:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763279.323323:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565857664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763279.323384:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565857792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763279.323433:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763279.323451:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763279/real 1789763279] req@ffff93f844cc8700 x1876699565857792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763279.323469:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565857792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763279.323480:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763279.323484:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763279.323488:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565857792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763279.323496:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565857920:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763279.323519:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763279.323532:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763279/real 1789763279] req@ffff93f844ccbb80 x1876699565857920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763279.323547:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565857920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763279.323569:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763279.323573:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763279.323577:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565857920:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763279.323583:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565858048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763279.323603:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763279.323615:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763279/real 1789763279] req@ffff93f844cc8000 x1876699565858048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763279.323629:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565858048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763279.323639:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763279.323691:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763279.323695:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565858048:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763279.386828:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763280.411024:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763281.434821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763282.458785:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763283.482893:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763284.442758:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763284.442765:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763284.442769:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763284.442779:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3219 00000100:00080000:3.0:1789763284.442789:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763284.442796:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763284.442842:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565858176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763284.442880:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763284.442884:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763284.442885:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763284.442889:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3219 00000100:00080000:3.0:1789763284.442892:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763284.442896:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763284.442905:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763284.442916:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565858176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763284.442920:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565858304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763284.442931:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763284.442935:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763284.442936:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763284.442939:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3219 00000100:00080000:3.0:1789763284.442942:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763284.442946:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763284.442949:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763284.442964:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565858432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763284.442971:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763284.442974:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763284.442978:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789763284.442979:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763284.442985:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3219 00000100:00080000:3.0:1789763284.442989:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763284.442994:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763284.443000:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763284.443016:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565858560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763284.443042:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763284/real 1789763284] req@ffff93f844d64000 x1876699565858176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763284.443061:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565858176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763284.443078:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763284.443082:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763284.443085:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565858176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763284.443117:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565858304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763284.443136:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763284.443147:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763284/real 1789763284] req@ffff93f844d64380 x1876699565858304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763284.443155:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565858304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763284.443161:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763284.443163:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763284.443165:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565858304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763284.443169:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565858432:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763284.443182:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763284.443187:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763284/real 1789763284] req@ffff93f844d65500 x1876699565858432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763284.443196:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565858432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763284.443201:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763284.443203:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763284.443205:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565858432:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763284.443209:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565858560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763284.443220:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763284.443226:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763284/real 1789763284] req@ffff93f844d66d80 x1876699565858560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763284.443235:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565858560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763284.443240:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763284.443242:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763284.443244:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565858560:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763284.506906:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763285.530924:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763286.554905:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763287.578863:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763288.602764:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763289.562736:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763289.562746:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763289.562752:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763289.562764:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3224 00000100:00080000:3.0:1789763289.562787:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763289.562795:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763289.562841:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565858688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763289.562866:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763289.562869:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763289.562870:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763289.562874:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3224 00000100:00080000:3.0:1789763289.562877:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763289.562880:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763289.562890:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763289.562898:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565858688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763289.562903:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565858816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763289.562912:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763289.562914:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763289.562916:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763289.562918:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3224 00000100:00080000:3.0:1789763289.562920:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763289.562923:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763289.562926:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763289.562934:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565858944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763289.562946:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763289.562949:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763289.562951:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763289.562953:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3224 00000100:00080000:3.0:1789763289.562955:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763289.562958:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763289.562961:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763289.562969:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565859072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763289.562976:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763289.563030:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763289/real 1789763289] req@ffff93f844d65c00 x1876699565858688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763289.563045:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565858688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763289.563055:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763289.563058:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763289.563061:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565858688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763289.563087:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565858816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763289.563104:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763289.563113:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763289/real 1789763289] req@ffff93f844d67480 x1876699565858816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763289.563122:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565858816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763289.563128:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763289.563130:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763289.563133:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565858816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763289.563136:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565858944:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763289.563147:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763289.563154:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763289/real 1789763289] req@ffff93f844d66a00 x1876699565858944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763289.563162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565858944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763289.563167:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763289.563168:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763289.563170:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565858944:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763289.563173:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565859072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763289.563184:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763289.563190:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763289/real 1789763289] req@ffff93f844d64e00 x1876699565859072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763289.563198:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565859072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763289.563208:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763289.563210:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763289.563212:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565859072:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763289.626843:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763290.650893:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763291.674869:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763292.698825:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763293.722845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763294.682806:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763294.682823:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763294.682828:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763294.682844:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3229 00000100:00080000:1.0:1789763294.682866:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763294.682880:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763294.682936:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565859200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763294.682971:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763294.682975:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763294.682977:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763294.682983:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3229 00000100:00080000:1.0:1789763294.682988:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763294.682994:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763294.683008:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763294.683030:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565859200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763294.683030:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565859328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763294.683064:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763294.683069:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763294.683072:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763294.683080:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3229 00000100:00080000:1.0:1789763294.683102:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763294.683107:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789763294.683112:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789763294.683125:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763294.683152:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565859456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763294.683160:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763294/real 1789763294] req@ffff93f844cc8380 x1876699565859200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763294.683176:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763294.683183:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:0.0:1789763294.683189:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565859200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763294.683191:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763294.683206:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3229 00000100:00080000:0.0:1789763294.683208:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763294.683213:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763294.683216:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:1.0:1789763294.683226:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763294.683237:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763294.683239:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565859200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763294.683264:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565859584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763294.683275:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565859328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763294.683309:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763294.683320:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763294/real 1789763294] req@ffff93f844cca300 x1876699565859328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763294.683330:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565859328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763294.683338:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763294.683341:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763294.683345:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565859328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763294.683352:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565859456:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763294.683376:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763294.683389:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763294/real 1789763294] req@ffff93f844cc9f80 x1876699565859456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763294.683401:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565859456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763294.683407:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763294.683409:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763294.683412:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565859456:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763294.683423:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565859584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763294.683439:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763294.683446:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763294/real 1789763294] req@ffff93f844cc9880 x1876699565859584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763294.683454:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565859584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763294.683459:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763294.683461:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763294.683463:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565859584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763294.746818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763295.770816:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763296.794781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763297.818837:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763298.842815:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763299.802774:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763299.802790:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763299.802798:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763299.802813:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3234 00000100:00080000:1.0:1789763299.802826:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763299.802836:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763299.802895:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565859712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763299.802926:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763299.802929:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763299.802931:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763299.802935:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3234 00000100:00080000:1.0:1789763299.802938:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763299.802942:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763299.802952:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763299.802964:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565859840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763299.802979:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565859712:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763299.802980:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763299.802997:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763299.803000:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763299.803005:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3234 00000100:00080000:1.0:1789763299.803008:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763299.803012:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763299.803017:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763299.803028:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565859968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763299.803042:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763299.803046:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763299.803048:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763299.803051:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3234 00000100:00080000:1.0:1789763299.803054:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763299.803058:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789763299.803066:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789763299.803068:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763299.803080:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565860096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763299.803111:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763299/real 1789763299] req@ffff93f844cc8a80 x1876699565859712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763299.803126:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565859712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763299.803135:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763299.803138:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763299.803141:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565859712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763299.803162:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565859840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763299.803177:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763299.803186:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763299/real 1789763299] req@ffff93f844cc9180 x1876699565859840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763299.803194:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565859840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763299.803199:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763299.803201:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763299.803203:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565859840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763299.803207:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565859968:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763299.803228:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763299.803236:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763299/real 1789763299] req@ffff93f844cc9880 x1876699565859968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763299.803243:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565859968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763299.803253:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763299.803255:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763299.803257:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565859968:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763299.803260:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565860096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763299.803271:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763299.803277:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763299/real 1789763299] req@ffff93f844cc9f80 x1876699565860096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763299.803284:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565860096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763299.803289:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763299.803291:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763299.803292:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565860096:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763299.866842:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763300.890834:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763301.914780:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763302.938795:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763303.962844:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763304.922732:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763304.922742:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763304.922746:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763304.922755:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3239 00000100:00080000:1.0:1789763304.922765:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763304.922772:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763304.922812:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565860224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763304.922830:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763304.922833:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763304.922834:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763304.922837:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3239 00000100:00080000:1.0:1789763304.922840:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763304.922842:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763304.922850:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763304.922859:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565860352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763304.922867:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763304.922869:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763304.922870:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763304.922873:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3239 00000100:00080000:1.0:1789763304.922875:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763304.922879:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763304.922882:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565860224:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789763304.922883:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763304.922893:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565860480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763304.922900:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763304.922903:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763304.922904:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763304.922906:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3239 00000100:00080000:1.0:1789763304.922908:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763304.922910:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763304.922914:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763304.922921:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565860608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763304.922950:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763304.922991:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763304/real 1789763304] req@ffff93f844cca300 x1876699565860224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763304.923006:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565860224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763304.923014:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763304.923017:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763304.923020:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565860224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763304.923026:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565860352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763304.923040:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763304.923049:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763304/real 1789763304] req@ffff93f844cc8380 x1876699565860352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763304.923057:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565860352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763304.923062:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763304.923064:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763304.923066:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565860352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763304.923084:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565860480:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763304.923095:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763304.923101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763304/real 1789763304] req@ffff93f844cc8000 x1876699565860480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763304.923108:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565860480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763304.923113:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763304.923115:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763304.923117:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565860480:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763304.923121:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565860608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763304.923131:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763304.923137:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763304/real 1789763304] req@ffff93f844ccbb80 x1876699565860608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763304.923145:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565860608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763304.923150:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763304.923152:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763304.923154:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565860608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763304.986842:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763306.010821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763307.034774:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763308.058788:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763309.082781:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763310.042765:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763310.042776:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763310.042780:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763310.042791:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3245 00000100:00080000:3.0:1789763310.042801:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763310.042809:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763310.042854:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565860736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763310.042878:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763310.042881:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763310.042883:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763310.042886:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3245 00000100:00080000:3.0:1789763310.042889:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763310.042892:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763310.042902:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763310.042914:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565860736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763310.042915:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565860864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763310.042930:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763310.042932:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763310.042934:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763310.042937:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3245 00000100:00080000:3.0:1789763310.042940:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763310.042942:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763310.042947:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763310.042957:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565860992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763310.042965:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763310.042968:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763310.042969:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763310.042972:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3245 00000100:00080000:3.0:1789763310.042993:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763310.042998:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763310.043005:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763310.043020:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565861120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763310.043028:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763310.043114:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763310/real 1789763310] req@ffff93f844d67100 x1876699565860736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763310.043162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565860736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763310.043177:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763310.043182:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763310.043187:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565860736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763310.043217:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565860864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763310.043237:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763310.043247:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763310/real 1789763310] req@ffff93f844d67800 x1876699565860864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763310.043256:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565860864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763310.043267:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763310.043270:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763310.043272:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565860864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763310.043276:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565860992:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763310.043289:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763310.043296:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763310/real 1789763310] req@ffff93f844d64700 x1876699565860992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763310.043304:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565860992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763310.043312:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763310.043316:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763310.043319:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565860992:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763310.043326:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565861120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763310.043347:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763310.043360:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763310/real 1789763310] req@ffff93f844d67b80 x1876699565861120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763310.043371:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565861120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763310.043377:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763310.043379:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763310.043382:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565861120:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763310.106784:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763311.130813:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763312.154835:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763313.178786:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763314.202767:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763315.162805:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763315.162817:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763315.162824:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763315.162837:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3250 00000100:00080000:1.0:1789763315.162848:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763315.162858:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763315.162912:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565861248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763315.162962:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763315.162968:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763315.162971:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763315.162977:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3250 00000100:00080000:1.0:1789763315.162983:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763315.162989:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763315.163002:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763315.163024:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565861376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763315.163040:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789763315.163046:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565861248:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763315.163047:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763315.163051:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763315.163063:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3250 00000100:00080000:1.0:1789763315.163069:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763315.163075:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763315.163091:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763315.163118:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565861504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763315.163155:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763315.163163:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763315.163167:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763315.163176:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3250 00000100:00080000:1.0:1789763315.163183:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763315.163189:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763315.163198:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763315.163219:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565861632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763315.163252:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565861376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763315.163320:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565861504:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763315.163357:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565861632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763315.226842:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763316.250787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763317.274763:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789763318.234930:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789763318.234989:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789763318.235013:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763318.235073:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763318.235084:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763318.235093:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789763318.235136:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763315/real 1789763318] req@ffff93f844ccb800 x1876699565861376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763318.235167:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565861376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763318.235181:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763318.235194:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763318.235200:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565861376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763318.235208:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763315/real 1789763318] req@ffff93f844ccb480 x1876699565861504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763318.235221:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565861504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763318.235230:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763318.235234:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763318.235239:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565861504:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763318.235250:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763315/real 1789763318] req@ffff93f844ccaa00 x1876699565861632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763318.235272:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565861632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763318.235285:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763318.235291:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763318.235298:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565861632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763318.235309:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763315/real 1789763318] req@ffff93f844cc8700 x1876699565861248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763318.235329:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565861248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763318.235342:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763318.235356:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763318.235363:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565861248:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789763318.298816:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763319.322849:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763320.282876:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763320.282886:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763320.282892:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763320.282905:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3255 00000100:00080000:3.0:1789763320.282916:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763320.282926:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763320.282969:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565861760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763320.283000:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763320.283006:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763320.283010:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763320.283021:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3255 00000100:00100000:2.0:1789763320.283045:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565861760:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763320.283050:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763320.283066:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763320.283083:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763320.283117:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565861888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763320.283139:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763320.283150:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763320.283161:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763320.283166:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763320.283178:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3255 00000100:00080000:3.0:1789763320.283188:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763320.283200:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763320.283210:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763320/real 1789763320] req@ffff93f844d65880 x1876699565861760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1789763320.283211:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763320.283250:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565862016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763320.283270:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565861760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763320.283280:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763320.283301:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763320.283305:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763320.283308:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763320.283324:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1789763320.283329:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3255 00000100:00100000:2.0:1789763320.283337:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565861760:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763320.283345:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763320.283367:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763320.283378:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763320.283384:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565861888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763320.283409:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565862144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763320.283439:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763320.283470:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763320/real 1789763320] req@ffff93f844d66680 x1876699565861888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763320.283496:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565861888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763320.283511:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763320.283516:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763320.283522:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565861888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763320.283533:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565862016:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763320.283601:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763320.283623:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763320/real 1789763320] req@ffff93f844d67b80 x1876699565862016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763320.283680:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565862016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763320.283710:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763320.283716:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763320.283721:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565862016:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763320.283750:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565862144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763320.283788:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763320.283816:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763320/real 1789763320] req@ffff93f844d64700 x1876699565862144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763320.283843:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565862144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763320.283860:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763320.283866:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763320.283872:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565862144:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763320.346823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763321.370783:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763322.394797:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763323.418829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763324.442867:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763325.402760:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763325.402774:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763325.402780:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763325.402794:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3260 00000100:00080000:3.0:1789763325.402807:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763325.402816:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763325.402874:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565862272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763325.402910:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763325.402916:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763325.402920:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763325.402927:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3260 00000100:00080000:3.0:1789763325.402934:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763325.402942:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763325.402958:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763325.402985:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565862400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763325.403007:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763325.403015:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763325.403020:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763325.403028:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3260 00000100:00080000:3.0:1789763325.403036:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789763325.403040:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565862272:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763325.403050:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763325.403066:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763325.403097:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565862528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763325.403124:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763325.403132:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:2.0:1789763325.403138:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763325.403141:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763325.403152:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3260 00000100:00080000:3.0:1789763325.403162:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763325.403170:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763325.403180:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763325.403199:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565862656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763325.403226:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763325/real 1789763325] req@ffff93f844d67800 x1876699565862272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763325.403256:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565862272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763325.403272:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763325.403279:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763325.403286:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565862272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763325.403296:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565862400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763325.403324:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763325.403340:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763325/real 1789763325] req@ffff93f844d67100 x1876699565862400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763325.403355:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565862400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763325.403365:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763325.403368:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763325.403372:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565862400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763325.403401:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565862528:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763325.403422:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763325.403433:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763325/real 1789763325] req@ffff93f844d64e00 x1876699565862528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763325.403448:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565862528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763325.403458:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763325.403462:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763325.403466:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565862528:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763325.403472:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565862656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763325.403490:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763325.403501:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763325/real 1789763325] req@ffff93f844d66a00 x1876699565862656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763325.403547:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565862656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763325.403557:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763325.403561:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763325.403565:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565862656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763325.466850:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763326.490882:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763327.514887:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763328.538931:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763329.562753:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763330.522778:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763330.522790:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763330.522796:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763330.522811:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3265 00000100:00080000:3.0:1789763330.522824:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763330.522833:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763330.522890:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565862784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763330.522919:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763330.522922:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763330.522924:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763330.522927:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3265 00000100:00080000:3.0:1789763330.522930:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763330.522935:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763330.522946:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763330.522957:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565862912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763330.522963:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565862784:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763330.522971:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763330.522975:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763330.522982:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763330.522986:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3265 00000100:00080000:3.0:1789763330.522988:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763330.522991:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763330.522996:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763330.523007:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565863040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763330.523015:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763330.523019:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763330.523020:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763330.523022:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3265 00000100:00080000:3.0:1789763330.523025:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763330.523028:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763330.523030:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763330.523038:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565863168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763330.523050:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763330.523101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763330/real 1789763330] req@ffff93f844d67480 x1876699565862784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763330.523119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565862784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763330.523131:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763330.523135:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763330.523139:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565862784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763330.523171:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565862912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763330.523187:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763330.523197:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763330/real 1789763330] req@ffff93f844d65c00 x1876699565862912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763330.523205:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565862912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763330.523211:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763330.523214:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763330.523216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565862912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763330.523220:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565863040:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763330.523231:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763330.523238:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763330/real 1789763330] req@ffff93f844d66d80 x1876699565863040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763330.523245:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565863040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763330.523251:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763330.523253:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763330.523255:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565863040:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763330.523259:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565863168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763330.523270:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763330.523277:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763330/real 1789763330] req@ffff93f844d65500 x1876699565863168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763330.523285:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565863168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763330.523290:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763330.523292:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763330.523294:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565863168:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763330.586846:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763331.610898:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763332.634879:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763333.658849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763334.682882:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763335.642809:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763335.642833:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763335.642840:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763335.642855:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3270 00000100:00080000:1.0:1789763335.642870:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763335.642880:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763335.642956:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565863296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763335.642989:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763335.642993:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763335.642996:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763335.643000:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3270 00000100:00080000:1.0:1789763335.643004:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763335.643009:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763335.643024:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763335.643037:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565863296:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763335.643038:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565863424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763335.643054:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763335.643057:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763335.643058:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763335.643061:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3270 00000100:00080000:1.0:1789763335.643063:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763335.643065:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763335.643069:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763335.643077:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565863552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763335.643084:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763335.643087:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763335.643088:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763335.643091:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3270 00000100:00080000:1.0:1789763335.643112:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763335.643115:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763335.643118:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789763335.643125:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763335.643129:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565863680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763335.643184:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763335/real 1789763335] req@ffff93f844cc9c00 x1876699565863296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763335.643204:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565863296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763335.643217:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763335.643222:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763335.643228:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565863296:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763335.643259:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565863424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763335.643281:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763335.643291:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763335/real 1789763335] req@ffff93f844cca680 x1876699565863424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763335.643300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565863424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763335.643312:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763335.643315:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763335.643317:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565863424:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763335.643321:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565863552:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763335.643334:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763335.643340:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763335/real 1789763335] req@ffff93f844cc8e00 x1876699565863552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763335.643348:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565863552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763335.643355:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763335.643356:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763335.643358:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565863552:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763335.643362:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565863680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763335.643373:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763335.643380:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763335/real 1789763335] req@ffff93f844ccb100 x1876699565863680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763335.643387:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565863680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763335.643392:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763335.643394:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763335.643397:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565863680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763335.706830:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763336.730888:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763337.754863:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763338.778824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763339.802896:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763340.762815:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763340.762832:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763340.762840:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763340.762858:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3275 00000100:00080000:1.0:1789763340.762888:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763340.762900:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763340.762963:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565863808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763340.763003:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763340.763009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763340.763012:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763340.763017:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3275 00000100:00080000:1.0:1789763340.763022:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763340.763028:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763340.763043:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763340.763059:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565863936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763340.763074:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763340.763079:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763340.763084:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789763340.763089:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565863808:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763340.763093:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3275 00000100:00080000:1.0:1789763340.763104:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763340.763110:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763340.763116:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763340.763132:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565864064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763340.763155:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763340.763160:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763340.763163:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763340.763169:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3275 00000100:00080000:1.0:1789763340.763193:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763340.763199:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763340.763205:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789763340.763214:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763340.763228:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565864192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763340.763294:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763340/real 1789763340] req@ffff93f844cc9500 x1876699565863808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763340.763325:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565863808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763340.763342:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763340.763349:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763340.763355:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565863808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763340.763391:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565863936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763340.763464:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763340.763485:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763340/real 1789763340] req@ffff93f844ccad80 x1876699565863936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763340.763511:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565863936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763340.763529:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763340.763535:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763340.763543:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565863936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763340.763556:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565864064:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763340.763600:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763340.763619:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763340/real 1789763340] req@ffff93f844ccb100 x1876699565864064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763340.763636:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565864064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763340.763682:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763340.763686:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763340.763691:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565864064:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763340.763698:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565864192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763340.763728:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763340.763743:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763340/real 1789763340] req@ffff93f844cc8e00 x1876699565864192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763340.763759:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565864192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763340.763769:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763340.763773:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763340.763777:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565864192:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763340.826891:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763341.850914:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763342.874910:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763343.898946:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763344.922978:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763345.882829:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763345.882841:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763345.882847:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763345.882863:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3280 00000100:00080000:1.0:1789763345.882877:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763345.882887:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763345.882953:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565864320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763345.883004:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763345.883012:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763345.883016:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763345.883025:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3280 00000100:00080000:1.0:1789763345.883033:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763345.883040:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763345.883069:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763345.883083:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565864448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763345.883095:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763345.883098:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763345.883100:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763345.883105:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3280 00000100:00080000:1.0:1789763345.883108:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763345.883112:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763345.883117:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763345.883121:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565864320:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763345.883150:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565864576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763345.883165:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763345.883170:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763345.883172:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763345.883177:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3280 00000100:00080000:1.0:1789763345.883183:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763345.883187:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763345.883194:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789763345.883250:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763345.883253:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565864704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763345.883332:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763345/real 1789763345] req@ffff93f844cca680 x1876699565864320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763345.883402:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565864320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763345.883420:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763345.883425:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763345.883431:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565864320:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763345.883442:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565864448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763345.883475:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763345.883495:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763345/real 1789763345] req@ffff93f844cc9c00 x1876699565864448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763345.883508:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565864448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763345.883516:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763345.883519:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763345.883522:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565864448:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763345.883545:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565864576:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763345.883577:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763345.883589:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763345/real 1789763345] req@ffff93f844cc8700 x1876699565864576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763345.883600:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565864576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763345.883608:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763345.883611:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763345.883613:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565864576:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763345.883619:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565864704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763345.883635:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763345.883701:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763345/real 1789763345] req@ffff93f844ccaa00 x1876699565864704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763345.883739:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565864704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763345.883749:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763345.883753:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763345.883757:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565864704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763345.946903:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763346.970925:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763347.994857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763349.018892:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763350.042997:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763351.002787:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763351.002798:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763351.002805:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763351.002817:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3285 00000100:00080000:1.0:1789763351.002828:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763351.002839:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763351.002892:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565864832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763351.002941:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763351.002945:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763351.002947:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763351.002952:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3285 00000100:00080000:1.0:1789763351.002957:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763351.002964:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763351.002975:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763351.002993:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565864832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763351.002997:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565864960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763351.003022:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763351.003027:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763351.003029:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763351.003034:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3285 00000100:00080000:1.0:1789763351.003038:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763351.003042:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763351.003049:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763351.003082:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565865088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763351.003110:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763351.003115:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763351.003123:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763351.003128:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763351.003138:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3285 00000100:00080000:1.0:1789763351.003147:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763351.003156:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763351.003166:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763351/real 1789763351] req@ffff93f844ccb480 x1876699565864832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1789763351.003167:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763351.003194:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565865216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763351.003200:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565864832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763351.003236:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763351.003242:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763351.003247:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565864832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763351.003279:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565864960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763351.003305:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763351.003318:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763351/real 1789763351] req@ffff93f844ccb800 x1876699565864960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763351.003373:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565864960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763351.003382:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763351.003385:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763351.003389:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565864960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763351.003396:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565865088:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763351.003419:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763351.003431:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763351/real 1789763351] req@ffff93f844ccbb80 x1876699565865088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763351.003443:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565865088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763351.003451:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763351.003454:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763351.003457:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565865088:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763351.003462:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565865216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763351.003478:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763351.003487:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763351/real 1789763351] req@ffff93f844cc8000 x1876699565865216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763351.003499:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565865216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763351.003514:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763351.003517:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763351.003520:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565865216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763351.066857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763352.090870:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763353.115012:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763354.138902:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763355.162830:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763356.122790:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763356.122799:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763356.122804:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763356.122816:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3291 00000100:00080000:2.0:1789763356.122828:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763356.122841:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763356.122885:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565865344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763356.122917:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763356.122920:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763356.122923:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763356.122930:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3291 00000100:00080000:2.0:1789763356.122937:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763356.122943:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763356.122946:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565865344:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789763356.122955:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763356.122975:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565865472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763356.122990:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763356.122993:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763356.122995:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763356.122999:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3291 00000100:00080000:2.0:1789763356.123003:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763356.123009:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789763356.123016:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1789763356.123021:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763356.123036:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565865600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763356.123044:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763356.123048:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763356.123050:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763356.123054:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3291 00000100:00080000:2.0:1789763356.123057:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763356.123061:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763356.123065:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763356.123077:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565865728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763356.123083:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763356/real 1789763356] req@ffff93f8517c2300 x1876699565865344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763356.123114:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565865344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763356.123124:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763356.123128:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763356.123131:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565865344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763356.123153:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565865472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763356.123174:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763356.123189:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763356/real 1789763356] req@ffff93f8517c1f80 x1876699565865472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763356.123203:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565865472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763356.123210:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763356.123213:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763356.123216:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565865472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763356.123224:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565865600:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763356.123246:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763356.123257:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763356/real 1789763356] req@ffff93f8517c3b80 x1876699565865600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763356.123272:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565865600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763356.123297:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763356.123303:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763356.123309:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565865600:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763356.123319:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565865728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763356.123338:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763356.123347:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763356/real 1789763356] req@ffff93f8517c0000 x1876699565865728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763356.123354:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565865728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763356.123359:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763356.123362:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763356.123363:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565865728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763356.186765:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763357.210979:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763358.234962:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763359.258916:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763360.283023:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763361.243454:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763361.243476:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763361.243485:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763361.243506:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3296 00000100:00080000:1.0:1789763361.243526:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763361.243541:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763361.243619:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565865856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763361.243721:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763361.243730:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763361.243734:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763361.243743:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3296 00000100:00080000:1.0:1789763361.243752:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763361.243762:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763361.243785:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763361.243814:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565865856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763361.243817:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565865984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763361.243855:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763361.243864:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763361.243869:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763361.243879:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3296 00000100:00080000:1.0:1789763361.243888:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763361.243897:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763361.243911:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763361.243939:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565866112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763361.243967:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763361.243975:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763361.243979:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789763361.243991:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763361.243996:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3296 00000100:00080000:1.0:1789763361.244005:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763361.244013:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763361.244022:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763361.244043:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565866240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763361.244077:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763361/real 1789763361] req@ffff93f844cc8380 x1876699565865856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763361.244122:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565865856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763361.244144:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763361.244153:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763361.244161:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565865856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763361.244212:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565865984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763361.244301:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763361.244332:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763361/real 1789763361] req@ffff93f844cca300 x1876699565865984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763361.244359:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565865984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763361.244377:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763361.244385:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763361.244392:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565865984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763361.244405:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565866112:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763361.244448:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763361.244473:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763361/real 1789763361] req@ffff93f844cc9f80 x1876699565866112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763361.244505:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565866112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763361.244533:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763361.244539:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763361.244546:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565866112:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763361.244557:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565866240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763361.244593:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763361.244614:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763361/real 1789763361] req@ffff93f844cc9880 x1876699565866240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763361.244635:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565866240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763361.244715:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763361.244725:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763361.244733:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565866240:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763361.306905:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763362.330926:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763363.354949:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763364.378878:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763365.402768:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763366.362730:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763366.362742:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763366.362748:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763366.362759:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3301 00000100:00080000:1.0:1789763366.362770:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763366.362779:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763366.362815:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565866368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763366.362842:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763366.362845:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763366.362847:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763366.362850:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3301 00000100:00080000:1.0:1789763366.362852:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763366.362855:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763366.362864:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763366.362874:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565866496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763366.362883:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763366.362885:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763366.362889:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789763366.362891:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565866368:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763366.362895:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3301 00000100:00080000:1.0:1789763366.362902:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763366.362906:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763366.362910:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763366.362921:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565866624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763366.362934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763366.362937:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763366.362939:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763366.362941:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3301 00000100:00080000:1.0:1789763366.362952:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763366.362955:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789763366.362961:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789763366.362962:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763366.362974:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565866752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763366.363005:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763366/real 1789763366] req@ffff93f844cc9180 x1876699565866368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763366.363022:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565866368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763366.363031:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763366.363035:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763366.363038:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565866368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763366.363057:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565866496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763366.363073:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763366.363082:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763366/real 1789763366] req@ffff93f844cc8a80 x1876699565866496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763366.363091:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565866496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763366.363097:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763366.363100:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763366.363102:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565866496:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763366.363106:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565866624:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763366.363118:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763366.363124:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763366/real 1789763366] req@ffff93f844cc9880 x1876699565866624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763366.363132:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565866624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763366.363138:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763366.363140:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763366.363142:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565866624:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763366.363146:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565866752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763366.363157:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763366.363164:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763366/real 1789763366] req@ffff93f844cc9f80 x1876699565866752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763366.363172:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565866752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763366.363178:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763366.363180:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763366.363182:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565866752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763366.426783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763367.450925:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763368.474854:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763369.498762:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763370.522760:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763371.482818:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763371.482836:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763371.482843:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763371.482861:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3306 00000100:00080000:1.0:1789763371.482879:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763371.482893:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763371.482944:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565866880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763371.482981:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763371.482986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763371.482988:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763371.482994:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3306 00000100:00080000:1.0:1789763371.482999:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763371.483005:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763371.483019:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763371.483035:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565867008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763371.483052:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565866880:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763371.483056:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763371.483069:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763371.483072:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763371.483078:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3306 00000100:00080000:1.0:1789763371.483082:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763371.483087:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763371.483093:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763371.483112:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565867136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763371.483126:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763371.483131:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763371.483134:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763371.483138:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3306 00000100:00080000:1.0:1789763371.483143:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763371.483152:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789763371.483153:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789763371.483161:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763371.483218:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565867264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763371.483253:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763371/real 1789763371] req@ffff93f844cca300 x1876699565866880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763371.483281:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565866880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763371.483296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763371.483302:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763371.483307:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565866880:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763371.483338:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565867008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763371.483370:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763371.483388:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763371/real 1789763371] req@ffff93f844cc8380 x1876699565867008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763371.483404:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565867008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763371.483424:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763371.483428:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763371.483432:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565867008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763371.483439:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565867136:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763371.483461:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763371.483479:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763371/real 1789763371] req@ffff93f844cc8000 x1876699565867136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763371.483503:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565867136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763371.483520:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763371.483527:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763371.483534:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565867136:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763371.483547:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565867264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763371.483586:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763371.483606:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763371/real 1789763371] req@ffff93f844ccbb80 x1876699565867264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763371.483621:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565867264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763371.483632:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763371.483637:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763371.483686:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565867264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763371.546822:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763372.570757:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763373.594854:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763374.618911:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763375.642900:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763376.602732:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763376.602740:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763376.602744:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763376.602753:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3311 00000100:00080000:0.0:1789763376.602761:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763376.602768:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763376.602801:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565867392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763376.602826:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763376.602829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763376.602830:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763376.602833:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3311 00000100:00080000:0.0:1789763376.602835:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763376.602838:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763376.602845:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763376.602854:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565867520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763376.602862:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763376.602864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763376.602866:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789763376.602871:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565867392:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763376.602872:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3311 00000100:00080000:0.0:1789763376.602877:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763376.602881:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763376.602885:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763376.602894:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565867648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763376.602901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763376.602903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763376.602905:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763376.602907:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3311 00000100:00080000:0.0:1789763376.602910:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763376.602912:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763376.602915:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763376.602922:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565867776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763376.602933:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763376.602973:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763376/real 1789763376] req@ffff93f850d41180 x1876699565867392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763376.602992:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565867392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763376.603000:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763376.603003:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763376.603006:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565867392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763376.603023:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565867520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763376.603038:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763376.603046:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763376/real 1789763376] req@ffff93f850d43100 x1876699565867520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763376.603054:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565867520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763376.603059:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763376.603062:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763376.603064:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565867520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763376.603067:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565867648:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763376.603078:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763376.603083:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763376/real 1789763376] req@ffff93f850d41f80 x1876699565867648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763376.603090:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565867648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763376.603099:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763376.603101:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763376.603104:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565867648:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763376.603107:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565867776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763376.603116:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763376.603122:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763376/real 1789763376] req@ffff93f850d41500 x1876699565867776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763376.603129:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565867776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763376.603150:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763376.603154:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763376.603168:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565867776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763376.666875:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763377.690863:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763378.714858:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763379.738826:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763380.762822:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763381.722820:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763381.722834:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763381.722842:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763381.722857:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3316 00000100:00080000:0.0:1789763381.722872:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763381.722885:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763381.722957:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565867904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763381.722997:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763381.723002:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763381.723006:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763381.723011:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3316 00000100:00080000:0.0:1789763381.723016:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763381.723023:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763381.723036:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763381.723056:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565868032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763381.723070:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565867904:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763381.723076:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763381.723090:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763381.723093:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763381.723131:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3316 00000100:00080000:0.0:1789763381.723149:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763381.723155:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763381.723164:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763381.723185:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565868160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763381.723199:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763381.723204:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763381.723207:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763381.723212:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3316 00000100:00080000:0.0:1789763381.723217:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763381.723222:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763381.723227:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763381.723243:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565868288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763381.723280:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565868032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763381.723334:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565868160:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763381.723375:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565868288:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763381.786947:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763382.810867:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763383.834929:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1789763384.795464:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1789763384.795530:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1789763384.795554:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789763384.795632:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789763384.795741:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789763384.795759:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789763384.795913:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763381/real 1789763384] req@ffff93f850d43480 x1876699565868032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763384.795958:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565868032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763384.795989:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763384.796000:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763384.796008:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565868032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763384.796021:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763381/real 1789763384] req@ffff93f850d41500 x1876699565868160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763384.796044:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565868160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763384.796057:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763384.796064:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763384.796069:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565868160:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763384.796109:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763381/real 1789763384] req@ffff93f850d41f80 x1876699565868288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763384.796132:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565868288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763384.796145:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763384.796151:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763384.796157:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565868288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763384.796169:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763381/real 1789763384] req@ffff93f850d43800 x1876699565867904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763384.796191:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565867904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763384.796205:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763384.796211:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763384.796218:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565867904:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789763384.858832:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763385.882892:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763386.842836:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763386.842846:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763386.842851:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763386.842862:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3321 00000100:00080000:3.0:1789763386.842872:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763386.842881:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763386.842918:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565868416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763386.842963:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763386.842967:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763386.842970:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763386.842975:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3321 00000100:00080000:3.0:1789763386.842980:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763386.842987:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763386.842999:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763386.843032:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565868416:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763386.843033:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565868544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763386.843076:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763386.843081:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763386.843082:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763386.843087:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3321 00000100:00080000:3.0:1789763386.843091:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763386.843095:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763386.843103:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763386.843124:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565868672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763386.843140:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763386.843146:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763386.843149:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763386.843156:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3321 00000800:00000100:0.0:1789763386.843166:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763386.843169:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763386.843180:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763386.843205:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763386.843231:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565868800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763386.843247:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763386/real 1789763386] req@ffff93f844d64380 x1876699565868416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763386.843277:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565868416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763386.843292:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763386.843299:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763386.843307:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565868416:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763386.843380:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565868544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763386.843418:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763386.843439:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763386/real 1789763386] req@ffff93f844d64000 x1876699565868544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763386.843469:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565868544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763386.843483:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763386.843489:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763386.843494:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565868544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763386.843505:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565868672:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763386.843538:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763386.843557:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763386/real 1789763386] req@ffff93f844d64a80 x1876699565868672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763386.843576:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565868672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763386.843589:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763386.843594:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763386.843599:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565868672:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763386.843608:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565868800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763386.843640:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763386.843690:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763386/real 1789763386] req@ffff93f844d65f80 x1876699565868800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763386.843708:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565868800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763386.843728:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763386.843732:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763386.843737:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565868800:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763386.906856:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763387.930860:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763388.954803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763389.978836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763391.002868:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763391.962714:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763391.962722:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763391.962726:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763391.962734:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3326 00000100:00080000:3.0:1789763391.962743:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763391.962750:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763391.962784:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565868928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763391.962805:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763391.962807:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763391.962809:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763391.962812:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3326 00000100:00080000:3.0:1789763391.962815:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763391.962818:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763391.962825:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763391.962836:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565868928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763391.962837:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565869056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763391.962851:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763391.962853:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763391.962854:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763391.962857:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3326 00000100:00080000:3.0:1789763391.962859:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763391.962862:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763391.962865:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763391.962874:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565869184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763391.962881:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763391.962883:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763391.962885:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763391.962887:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3326 00000100:00080000:3.0:1789763391.962890:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763391.962892:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763391.962895:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789763391.962897:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763391.962905:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565869312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763391.962943:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763391/real 1789763391] req@ffff93f844d66300 x1876699565868928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763391.962961:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565868928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763391.962970:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763391.962974:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763391.962978:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565868928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763391.962998:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565869056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763391.963017:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763391.963048:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763391/real 1789763391] req@ffff93f844d65180 x1876699565869056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763391.963060:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565869056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763391.963067:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763391.963070:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763391.963074:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565869056:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763391.963080:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565869184:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763391.963098:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763391.963105:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763391/real 1789763391] req@ffff93f844d65f80 x1876699565869184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763391.963112:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565869184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763391.963118:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763391.963120:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763391.963122:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565869184:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763391.963129:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565869312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763391.963143:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763391.963160:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763391/real 1789763391] req@ffff93f844d64a80 x1876699565869312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763391.963171:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565869312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763391.963180:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763391.963183:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763391.963186:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565869312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763392.026900:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763393.050847:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763394.074908:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763395.098948:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763396.123035:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763397.082737:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763397.082776:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763397.082784:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763397.082803:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3332 00000100:00080000:3.0:1789763397.082830:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763397.082842:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763397.082913:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565869440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763397.082954:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763397.082959:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763397.082961:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763397.082967:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3332 00000100:00080000:3.0:1789763397.082971:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763397.082975:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763397.083005:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763397.083051:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565869568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763397.083067:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763397.083071:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763397.083073:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763397.083084:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3332 00000100:00100000:0.0:1789763397.083092:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565869440:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763397.083093:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763397.083113:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763397.083123:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763397.083149:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565869696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763397.083172:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763397.083176:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763397.083178:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763397.083182:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3332 00000100:00080000:3.0:1789763397.083189:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763397.083194:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763397.083200:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789763397.083208:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763397.083218:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565869824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763397.083295:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763397/real 1789763397] req@ffff93f844d64000 x1876699565869440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763397.083347:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565869440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763397.083358:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763397.083362:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763397.083367:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565869440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763397.083394:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565869568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763397.083416:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763397.083427:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763397/real 1789763397] req@ffff93f844d64380 x1876699565869568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763397.083437:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565869568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763397.083444:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763397.083447:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763397.083450:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565869568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763397.083455:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565869696:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763397.083468:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763397.083476:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763397/real 1789763397] req@ffff93f844d65500 x1876699565869696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763397.083485:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565869696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763397.083498:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763397.083500:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763397.083503:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565869696:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763397.083508:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565869824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763397.083521:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763397.083528:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763397/real 1789763397] req@ffff93f844d66d80 x1876699565869824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763397.083537:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565869824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763397.083543:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763397.083546:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763397.083548:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565869824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763397.146893:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763398.170962:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763399.194889:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763400.219013:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763401.243036:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763402.202721:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763402.202729:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763402.202733:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763402.202741:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3337 00000100:00080000:3.0:1789763402.202751:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763402.202758:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763402.202792:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565869952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763402.202817:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763402.202821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763402.202825:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763402.202829:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3337 00000100:00080000:3.0:1789763402.202834:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763402.202838:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763402.202846:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763402.202862:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565869952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763402.202867:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565870080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763402.202878:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763402.202882:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763402.202884:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763402.202888:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3337 00000100:00080000:3.0:1789763402.202890:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763402.202892:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763402.202896:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763402.202903:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565870208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763402.202910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763402.202913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763402.202914:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763402.202916:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3337 00000100:00080000:3.0:1789763402.202919:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763402.202922:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789763402.202927:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1789763402.202929:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763402.202937:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565870336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763402.203052:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763402/real 1789763402] req@ffff93f844d65c00 x1876699565869952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763402.203067:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565869952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763402.203075:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763402.203078:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763402.203081:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565869952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763402.203100:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565870080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763402.203117:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763402.203126:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763402/real 1789763402] req@ffff93f844d67480 x1876699565870080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763402.203134:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565870080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763402.203140:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763402.203142:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763402.203144:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565870080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763402.203148:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565870208:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763402.203159:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763402.203165:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763402/real 1789763402] req@ffff93f844d66a00 x1876699565870208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763402.203173:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565870208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763402.203178:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763402.203179:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763402.203181:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565870208:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763402.203185:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565870336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763402.203195:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763402.203201:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763402/real 1789763402] req@ffff93f844d64e00 x1876699565870336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763402.203209:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565870336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763402.203214:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763402.203215:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763402.203217:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565870336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763402.266876:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763403.290923:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763404.314907:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763405.338773:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763406.362934:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763407.322866:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763407.322882:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763407.322890:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763407.322944:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3342 00000100:00080000:2.0:1789763407.322965:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763407.322977:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763407.323058:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565870464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763407.323115:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763407.323123:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763407.323127:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763407.323135:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3342 00000100:00080000:2.0:1789763407.323144:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763407.323153:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763407.323172:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763407.323180:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565870464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763407.323207:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565870592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763407.323232:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763407.323237:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763407.323240:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763407.323246:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3342 00000100:00080000:2.0:1789763407.323254:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763407.323265:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789763407.323267:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1789763407.323289:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763407.323312:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565870720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763407.323334:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789763407.323337:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763407/real 1789763407] req@ffff93f8517c0000 x1876699565870464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763407.323361:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763407.323371:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763407.323390:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3342 00000100:00080000:2.0:1789763407.323407:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763407.323421:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763407.323427:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565870464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1789763407.323436:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00080000:0.0:1789763407.323469:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1789763407.323480:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565870848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763407.323484:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763407.323514:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565870464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763407.323570:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565870592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763407.323619:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763407.323634:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763407/real 1789763407] req@ffff93f8517c3b80 x1876699565870592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763407.323648:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565870592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763407.323663:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763407.323667:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763407.323670:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565870592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763407.323732:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565870720:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763407.323764:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763407.323779:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763407/real 1789763407] req@ffff93f8517c1f80 x1876699565870720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763407.323794:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565870720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763407.323805:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763407.323809:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763407.323813:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565870720:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763407.323821:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565870848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763407.323841:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763407.323852:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763407/real 1789763407] req@ffff93f8517c2300 x1876699565870848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763407.323866:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565870848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763407.323875:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763407.323878:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763407.323883:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565870848:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763407.386926:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763408.410819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763409.434873:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763410.458997:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763411.482841:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763412.442814:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763412.442823:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763412.442827:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763412.442836:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3347 00000100:00080000:3.0:1789763412.442845:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763412.442852:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763412.442925:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565870976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763412.442961:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763412.442966:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763412.442969:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763412.442975:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3347 00000100:00080000:3.0:1789763412.442979:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763412.442986:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763412.442999:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763412.443019:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565870976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763412.443020:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565871104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763412.443044:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763412.443050:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763412.443053:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763412.443074:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3347 00000100:00080000:3.0:1789763412.443077:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763412.443081:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763412.443087:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763412.443097:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565871232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763412.443107:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763412.443113:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:0.0:1789763412.443116:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763412.443117:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763412.443125:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3347 00000100:00080000:3.0:1789763412.443129:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763412.443134:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763412.443138:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763412.443149:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565871360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763412.443167:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763412/real 1789763412] req@ffff93f844d67100 x1876699565870976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763412.443192:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565870976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763412.443203:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763412.443207:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763412.443211:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565870976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763412.443233:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565871104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763412.443253:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763412.443264:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763412/real 1789763412] req@ffff93f844d67800 x1876699565871104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763412.443274:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565871104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763412.443281:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763412.443284:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763412.443287:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565871104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763412.443291:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565871232:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763412.443305:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763412.443313:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763412/real 1789763412] req@ffff93f844d64700 x1876699565871232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763412.443323:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565871232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763412.443329:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763412.443332:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763412.443334:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565871232:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763412.443338:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565871360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763412.443351:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763412.443359:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763412/real 1789763412] req@ffff93f844d67b80 x1876699565871360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763412.443368:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565871360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763412.443374:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763412.443376:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763412.443379:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565871360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763412.506896:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763413.530991:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763414.554818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763415.578976:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763416.602891:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763417.562781:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763417.562798:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763417.562807:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763417.562828:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3352 00000100:00080000:3.0:1789763417.562873:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763417.562911:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763417.562958:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565871488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763417.562996:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763417.563001:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763417.563004:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763417.563010:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3352 00000100:00080000:3.0:1789763417.563016:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763417.563024:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763417.563036:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763417.563058:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565871616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763417.563087:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565871488:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763417.563119:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763417.563128:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763417.563135:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763417.563149:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3352 00000100:00080000:3.0:1789763417.563158:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763417.563167:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763417.563181:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763417.563210:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565871744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763417.563231:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763417.563239:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763417.563250:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763417.563257:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763417.563268:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3352 00000100:00080000:3.0:1789763417.563281:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763417.563291:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763417.563304:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763417.563317:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763417/real 1789763417] req@ffff93f844d66680 x1876699565871488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763417.563340:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565871872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763417.563350:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565871488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763417.563370:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763417.563395:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763417.563412:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565871488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763417.563471:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565871616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763417.563504:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763417.563520:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763417/real 1789763417] req@ffff93f844d65880 x1876699565871616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763417.563536:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565871616/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763417.563547:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763417.563551:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763417.563555:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565871616:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763417.563563:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565871744:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763417.563584:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763417.563596:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763417/real 1789763417] req@ffff93f844d67b80 x1876699565871744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763417.563610:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565871744/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763417.563621:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763417.563624:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763417.563628:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565871744:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763417.563636:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565871872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763417.563697:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763417.563713:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763417/real 1789763417] req@ffff93f844d64700 x1876699565871872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763417.563728:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565871872/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763417.563748:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763417.563752:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763417.563756:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565871872:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763417.626887:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763418.650896:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763419.674880:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763420.698779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763421.722881:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763422.682899:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763422.682915:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763422.682924:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763422.682942:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3357 00000100:00080000:1.0:1789763422.682959:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763422.682972:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763422.683027:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565872000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763422.683063:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763422.683071:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763422.683075:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763422.683083:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3357 00000100:00080000:1.0:1789763422.683107:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763422.683118:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763422.683135:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763422.683140:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565872000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763422.683167:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565872128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763422.683186:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763422.683191:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763422.683195:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763422.683201:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3357 00000100:00080000:1.0:1789763422.683219:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763422.683225:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789763422.683233:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789763422.683239:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763422.683260:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565872256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763422.683277:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763422.683282:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763422.683284:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763422.683289:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3357 00000100:00080000:1.0:1789763422.683296:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763422.683303:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763422.683309:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763422.683314:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763422/real 1789763422] req@ffff93f844ccb800 x1876699565872000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763422.683329:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565872384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763422.683354:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565872000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763422.683372:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763422.683379:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763422.683386:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565872000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763422.683430:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565872128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763422.683460:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763422.683476:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763422/real 1789763422] req@ffff93f844ccb480 x1876699565872128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763422.683494:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565872128/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763422.683504:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763422.683508:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763422.683513:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565872128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763422.683521:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565872256:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763422.683557:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763422.683580:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763422/real 1789763422] req@ffff93f844ccaa00 x1876699565872256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763422.683606:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565872256/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763422.683623:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763422.683631:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763422.683637:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565872256:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763422.683677:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565872384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763422.683709:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763422.683726:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763422/real 1789763422] req@ffff93f844cc8700 x1876699565872384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763422.683741:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565872384/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763422.683760:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763422.683764:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763422.683768:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565872384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763422.746871:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763423.770845:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763424.794905:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763425.818872:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763426.842872:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763427.802867:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763427.802884:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763427.802891:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763427.802908:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3362 00000100:00080000:2.0:1789763427.802926:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763427.802942:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763427.803001:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565872512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763427.803088:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763427.803110:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763427.803115:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763427.803124:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3362 00000100:00080000:2.0:1789763427.803133:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763427.803143:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763427.803160:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763427.803188:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565872640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763427.803210:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763427.803218:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763427.803222:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763427.803230:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3362 00000100:00080000:2.0:1789763427.803238:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763427.803246:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763427.803257:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763427.803285:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565872768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763427.803322:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763427.803331:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763427.803342:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763427.803379:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3362 00000100:00080000:2.0:1789763427.803387:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763427.803394:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763427.803403:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763427.803424:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565872896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763427.803586:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565872512:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1789763427.803698:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763427.803770:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763427/real 1789763427] req@ffff93f8517c0380 x1876699565872512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763427.803802:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565872512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763427.803820:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763427.803827:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763427.803835:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565872512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763427.803845:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565872640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763427.803900:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763427.803922:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763427/real 1789763427] req@ffff93f8517c1500 x1876699565872640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763427.803958:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565872640/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763427.803970:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763427.803976:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763427.803981:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565872640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763427.803991:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565872768:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763427.804025:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763427.804047:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763427/real 1789763427] req@ffff93f8517c2680 x1876699565872768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763427.804065:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565872768/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763427.804077:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763427.804082:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763427.804106:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565872768:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763427.804123:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565872896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763427.804177:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763427.804212:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763427/real 1789763427] req@ffff93f8517c0a80 x1876699565872896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763427.804247:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565872896/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763427.804260:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763427.804266:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763427.804273:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565872896:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763427.866820:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763428.890878:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763429.914881:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763430.938899:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763431.962975:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763432.922885:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763432.922902:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763432.922910:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763432.922929:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3367 00000100:00080000:2.0:1789763432.922947:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763432.922961:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763432.923027:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565873024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763432.923067:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763432.923073:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763432.923076:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763432.923083:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3367 00000100:00080000:2.0:1789763432.923087:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763432.923093:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763432.923108:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763432.923128:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565873152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763432.923148:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565873024:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763432.923154:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763432.923174:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763432.923178:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763432.923207:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3367 00000100:00080000:2.0:1789763432.923214:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763432.923219:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763432.923228:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763432.923246:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565873280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763432.923266:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1789763432.923268:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763432.923286:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763432.923291:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763432.923299:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3367 00000100:00080000:2.0:1789763432.923304:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763432.923310:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763432.923319:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763432.923336:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565873408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763432.923385:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763432/real 1789763432] req@ffff93f8517c2a00 x1876699565873024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763432.923413:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565873024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763432.923431:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763432.923437:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763432.923445:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565873024:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763432.923478:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565873152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763432.923510:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763432.923527:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763432/real 1789763432] req@ffff93f8517c1180 x1876699565873152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763432.923543:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565873152/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763432.923554:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763432.923558:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763432.923562:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565873152:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763432.923572:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565873280:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763432.923594:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763432.923606:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763432/real 1789763432] req@ffff93f8517c3100 x1876699565873280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763432.923621:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565873280/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763432.923631:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763432.923634:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763432.923638:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565873280:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763432.923681:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565873408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763432.923706:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763432.923758:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763432/real 1789763432] req@ffff93f8517c2d80 x1876699565873408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763432.923776:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565873408/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763432.923787:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763432.923791:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763432.923795:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565873408:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763432.986911:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763434.010932:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763435.034893:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763436.058867:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763437.082784:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763438.042782:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763438.042795:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763438.042801:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763438.042815:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3373 00000100:00080000:2.0:1789763438.042830:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763438.042841:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763438.042888:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565873536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763438.042924:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763438.042933:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763438.042935:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763438.042940:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3373 00000100:00080000:2.0:1789763438.042945:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763438.042951:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763438.042963:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763438.042987:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565873536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763438.042995:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565873664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763438.043013:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763438.043018:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763438.043020:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763438.043026:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3373 00000100:00080000:2.0:1789763438.043030:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763438.043035:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763438.043041:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763438.043057:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565873792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763438.043070:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763438.043085:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:0.0:1789763438.043091:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763438.043091:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763438.043106:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3373 00000100:00080000:2.0:1789763438.043111:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763438.043117:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763438.043124:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763438.043139:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565873920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763438.043201:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763438/real 1789763438] req@ffff93f8517c1880 x1876699565873536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763438.043244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565873536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763438.043264:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763438.043269:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763438.043275:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565873536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763438.043315:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565873664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763438.043365:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763438.043394:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763438/real 1789763438] req@ffff93f8517c3480 x1876699565873664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763438.043416:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565873664/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763438.043428:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763438.043432:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763438.043436:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565873664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763438.043445:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565873792:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763438.043474:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763438.043488:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763438/real 1789763438] req@ffff93f8517c3800 x1876699565873792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763438.043502:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565873792/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763438.043521:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763438.043525:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763438.043529:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565873792:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763438.043536:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565873920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763438.043555:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763438.043566:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763438/real 1789763438] req@ffff93f8517c0e00 x1876699565873920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763438.043580:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565873920/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763438.043590:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763438.043593:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763438.043597:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565873920:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763438.106887:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763439.130884:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763440.154855:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763441.178857:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763442.202851:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763443.162812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763443.162825:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763443.162831:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763443.162844:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3378 00000100:00080000:2.0:1789763443.162858:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763443.162868:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763443.162917:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565874048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763443.162991:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763443.162997:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763443.163000:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763443.163006:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3378 00000100:00080000:2.0:1789763443.163012:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763443.163019:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763443.163032:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763443.163057:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565874176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763443.163064:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565874048:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763443.163088:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763443.163091:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763443.163094:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763443.163100:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3378 00000100:00080000:2.0:1789763443.163106:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763443.163113:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763443.163123:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763443.163143:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565874304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763443.163159:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763443.163167:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763443.163175:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763443.163179:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763443.163188:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3378 00000100:00080000:2.0:1789763443.163195:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763443.163200:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763443.163208:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763443.163227:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565874432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763443.163238:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763443/real 1789763443] req@ffff93f8517c0e00 x1876699565874048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763443.163268:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565874048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763443.163281:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763443.163286:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763443.163291:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565874048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763443.163318:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565874176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763443.163345:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763443.163359:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763443/real 1789763443] req@ffff93f8517c3800 x1876699565874176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763443.163388:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565874176/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763443.163397:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763443.163400:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763443.163404:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565874176:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763443.163410:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565874304:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763443.163429:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763443.163440:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763443/real 1789763443] req@ffff93f8517c3480 x1876699565874304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763443.163452:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565874304/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763443.163460:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763443.163462:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763443.163465:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565874304:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763443.163471:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565874432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763443.163487:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763443.163497:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763443/real 1789763443] req@ffff93f8517c1880 x1876699565874432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763443.163508:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565874432/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763443.163516:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763443.163519:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763443.163522:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565874432:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763443.226886:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763444.250886:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763445.274900:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763446.298864:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763447.322862:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763448.282762:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763448.282775:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763448.282781:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763448.282793:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3383 00000100:00080000:2.0:1789763448.282806:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763448.282817:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763448.282875:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565874560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763448.282925:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763448.282934:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763448.282937:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763448.282944:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3383 00000100:00080000:2.0:1789763448.282951:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763448.282959:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763448.282972:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763448.282987:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565874560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763448.282998:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565874688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763448.283016:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763448.283021:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763448.283024:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763448.283029:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3383 00000100:00080000:2.0:1789763448.283034:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763448.283039:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763448.283046:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763448.283060:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565874816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763448.283074:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763448.283078:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763448.283081:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763448.283085:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3383 00000100:00080000:2.0:1789763448.283091:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763448.283096:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763448.283103:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763448.283119:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565874944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763448.283158:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565874688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763448.283205:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565874816:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763448.283237:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565874944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763448.346865:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763449.370952:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763450.394822:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789763451.354992:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789763451.355058:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789763451.355086:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763451.355165:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763451.355200:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763451.355218:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789763451.355298:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763448/real 1789763451] req@ffff93f8517c3100 x1876699565874688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763451.355339:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565874688/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763451.355359:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763451.355368:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763451.355375:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565874688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763451.355389:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763448/real 1789763451] req@ffff93f8517c1180 x1876699565874816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763451.355411:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565874816/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763451.355427:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763451.355433:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763451.355439:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565874816:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763451.355450:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763448/real 1789763451] req@ffff93f8517c2a00 x1876699565874944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763451.355479:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565874944/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763451.355491:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763451.355497:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763451.355502:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565874944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763451.355512:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763448/real 1789763451] req@ffff93f8517c2d80 x1876699565874560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763451.355543:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565874560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763451.355555:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763451.355560:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763451.355565:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565874560:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789763451.418818:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763452.442859:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763453.402825:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763453.402841:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763453.402849:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763453.402866:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3388 00000100:00080000:3.0:1789763453.402884:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763453.402898:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763453.402960:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565875072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763453.403012:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763453.403018:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763453.403021:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763453.403027:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3388 00000100:00080000:3.0:1789763453.403034:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763453.403040:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763453.403053:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763453.403072:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565875200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763453.403088:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763453.403096:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763453.403100:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763453.403109:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3388 00000100:00080000:3.0:1789763453.403117:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763453.403126:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763453.403137:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763453.403147:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565875072:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763453.403169:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565875328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763453.403195:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763453.403203:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763453.403208:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763453.403214:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3388 00000100:00080000:3.0:1789763453.403223:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763453.403228:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763453.403236:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763453.403260:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565875456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763453.403263:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763453.403345:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763453/real 1789763453] req@ffff93f844d67800 x1876699565875072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763453.403376:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565875072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763453.403397:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763453.403406:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763453.403416:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565875072:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763453.403431:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565875200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763453.403474:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763453.403501:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763453/real 1789763453] req@ffff93f844d67100 x1876699565875200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763453.403525:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565875200/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763453.403543:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763453.403549:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763453.403556:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565875200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763453.403627:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565875328:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763453.403737:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763453.403764:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763453/real 1789763453] req@ffff93f844d64e00 x1876699565875328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763453.403787:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565875328/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763453.403813:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763453.403819:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763453.403825:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565875328:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763453.403837:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565875456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763453.403877:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763453.403900:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763453/real 1789763453] req@ffff93f844d66a00 x1876699565875456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763453.403925:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565875456/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763453.403942:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763453.403948:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763453.403954:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565875456:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763453.466846:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763454.490864:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763455.514837:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763456.538911:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763457.562909:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763458.522856:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763458.522868:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763458.522873:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763458.522885:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3393 00000100:00080000:2.0:1789763458.522897:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763458.522917:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763458.522963:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565875584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763458.523010:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763458.523015:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763458.523017:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763458.523022:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3393 00000100:00080000:2.0:1789763458.523026:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763458.523030:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763458.523052:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763458.523073:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565875712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763458.523094:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763458.523112:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763458.523117:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763458.523124:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3393 00000100:00080000:2.0:1789763458.523163:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763458.523169:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763458.523180:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763458.523198:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565875584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763458.523210:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565875840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763458.523236:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763458.523243:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763458.523247:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763458.523255:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3393 00000100:00080000:2.0:1789763458.523271:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763458.523276:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763458.523283:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763458.523304:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565875968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763458.523338:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763458.523444:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763458/real 1789763458] req@ffff93f8517c0a80 x1876699565875584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763458.523484:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565875584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763458.523501:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763458.523507:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763458.523513:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565875584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763458.523527:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565875712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763458.523630:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763458.523654:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763458/real 1789763458] req@ffff93f8517c2680 x1876699565875712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763458.523770:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565875712/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763458.523791:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763458.523795:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763458.523798:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565875712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763458.523846:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565875840:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763458.523873:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763458.523886:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763458/real 1789763458] req@ffff93f8517c1500 x1876699565875840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763458.523898:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565875840/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763458.523907:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763458.523910:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763458.523913:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565875840:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763458.523919:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565875968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763458.523935:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763458.523945:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763458/real 1789763458] req@ffff93f8517c0380 x1876699565875968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763458.523957:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565875968/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763458.523964:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763458.523967:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763458.523969:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565875968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763458.586944:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763459.610776:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763460.634856:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763461.658811:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763462.682825:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763463.642815:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763463.642829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763463.642835:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763463.642850:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3398 00000100:00080000:3.0:1789763463.642866:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763463.642879:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763463.642930:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565876096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763463.642973:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763463.642979:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763463.642982:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763463.642988:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3398 00000100:00080000:3.0:1789763463.642993:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763463.642999:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763463.643013:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763463.643039:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565876096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763463.643042:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565876224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763463.643069:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763463.643076:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763463.643080:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763463.643090:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3398 00000100:00080000:3.0:1789763463.643099:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763463.643108:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763463.643121:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:1.0:1789763463.643159:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763463.643168:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565876352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763463.643194:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763463.643201:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763463.643206:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763463.643216:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3398 00000100:00080000:3.0:1789763463.643230:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763463.643243:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763463.643259:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763463/real 1789763463] req@ffff93f844d67480 x1876699565876096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1789763463.643263:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763463.643302:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565876480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763463.643317:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565876096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763463.643340:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763463.643347:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763463.643354:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565876096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763463.643411:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565876224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763463.643459:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763463.643482:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763463/real 1789763463] req@ffff93f844d65c00 x1876699565876224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763463.643536:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565876224/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763463.643549:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763463.643554:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763463.643558:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565876224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763463.643572:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565876352:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763463.643621:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763463.643689:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763463/real 1789763463] req@ffff93f844d66d80 x1876699565876352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763463.643709:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565876352/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763463.643721:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763463.643726:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763463.643737:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565876352:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763463.643745:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565876480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763463.643781:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763463.643797:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763463/real 1789763463] req@ffff93f844d65500 x1876699565876480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763463.643816:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565876480/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763463.643828:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763463.643833:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763463.643838:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565876480:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763463.706820:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763464.730787:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763465.754913:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763466.778839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763467.802886:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763468.762718:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763468.762727:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763468.762732:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763468.762743:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3403 00000100:00080000:3.0:1789763468.762753:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763468.762760:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763468.762809:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565876608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763468.762834:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763468.762836:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763468.762838:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763468.762841:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3403 00000100:00080000:3.0:1789763468.762843:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763468.762846:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763468.762854:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763468.762865:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565876736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763468.762882:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763468.762885:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763468.762888:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1789763468.762889:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565876608:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763468.762893:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3403 00000100:00080000:3.0:1789763468.762898:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763468.762900:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763468.762904:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763468.762913:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565876864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763468.762920:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763468.762923:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763468.762924:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763468.762927:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3403 00000100:00080000:3.0:1789763468.762929:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763468.762932:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763468.762935:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763468.762941:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565876992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763468.762949:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763468.762986:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763468/real 1789763468] req@ffff93f844d64380 x1876699565876608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763468.762999:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565876608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763468.763008:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763468.763010:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763468.763013:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565876608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763468.763032:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565876736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763468.763046:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763468.763054:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763468/real 1789763468] req@ffff93f844d64000 x1876699565876736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763468.763062:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565876736/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763468.763068:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763468.763070:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763468.763072:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565876736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763468.763075:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565876864:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763468.763086:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763468.763092:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763468/real 1789763468] req@ffff93f844d64a80 x1876699565876864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763468.763099:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565876864/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763468.763105:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763468.763106:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763468.763108:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565876864:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763468.763112:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565876992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763468.763122:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763468.763127:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763468/real 1789763468] req@ffff93f844d65f80 x1876699565876992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763468.763134:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565876992/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763468.763139:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763468.763141:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763468.763143:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565876992:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763468.826766:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763469.850859:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763470.874874:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763471.898886:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763472.922905:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763473.882782:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763473.882796:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763473.882802:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763473.882818:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3408 00000100:00080000:3.0:1789763473.882835:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763473.882849:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763473.882946:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565877120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763473.882980:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763473.882984:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763473.882987:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763473.882992:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3408 00000100:00080000:3.0:1789763473.882998:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763473.883004:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763473.883015:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763473.883033:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565877120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763473.883036:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565877248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763473.883057:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763473.883060:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763473.883062:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763473.883066:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3408 00000100:00080000:3.0:1789763473.883070:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763473.883073:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763473.883079:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763473.883094:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565877376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763473.883107:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763473.883111:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763473.883113:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1789763473.883120:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763473.883122:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3408 00000100:00080000:3.0:1789763473.883130:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763473.883135:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763473.883141:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763473.883153:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565877504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763473.883204:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763473/real 1789763473] req@ffff93f844d65180 x1876699565877120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763473.883238:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565877120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763473.883256:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763473.883261:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763473.883265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565877120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763473.883295:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565877248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763473.883329:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763473.883345:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763473/real 1789763473] req@ffff93f844d66300 x1876699565877248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763473.883359:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565877248/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763473.883369:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763473.883372:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763473.883376:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565877248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763473.883382:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565877376:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763473.883400:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763473.883411:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763473/real 1789763473] req@ffff93f844d65f80 x1876699565877376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763473.883452:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565877376/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763473.883462:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763473.883466:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763473.883469:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565877376:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763473.883476:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565877504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763473.883501:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763473.883513:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763473/real 1789763473] req@ffff93f844d64a80 x1876699565877504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763473.883534:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565877504/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763473.883542:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763473.883545:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763473.883549:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565877504:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763473.946852:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763474.970904:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763475.994806:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763477.018875:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763478.042812:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763479.002737:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763479.002747:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763479.002752:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763479.002779:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3413 00000100:00080000:3.0:1789763479.002787:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763479.002794:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763479.002831:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565877632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763479.002856:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763479.002859:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763479.002860:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763479.002864:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3413 00000100:00080000:3.0:1789763479.002867:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763479.002871:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763479.002879:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763479.002886:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565877632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763479.002892:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565877760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763479.002901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763479.002903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763479.002905:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763479.002908:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3413 00000100:00080000:3.0:1789763479.002910:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763479.002912:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763479.002915:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763479.002923:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565877888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763479.002930:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763479.002933:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763479.002934:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763479.002936:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3413 00000100:00080000:3.0:1789763479.002938:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763479.002941:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1789763479.002944:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1789763479.002945:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763479.002955:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565878016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763479.002985:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763479/real 1789763479] req@ffff93f844d64000 x1876699565877632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763479.002998:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565877632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763479.003006:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763479.003009:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763479.003012:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565877632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763479.003030:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565877760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763479.003055:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763479.003063:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763479/real 1789763479] req@ffff93f844d64380 x1876699565877760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763479.003071:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565877760/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763479.003077:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763479.003079:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763479.003081:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565877760:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763479.003085:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565877888:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763479.003095:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763479.003101:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763479/real 1789763479] req@ffff93f844d65500 x1876699565877888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763479.003108:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565877888/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763479.003123:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763479.003125:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763479.003127:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565877888:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763479.003130:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565878016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763479.003143:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763479.003154:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763479/real 1789763479] req@ffff93f844d66d80 x1876699565878016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763479.003165:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565878016/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763479.003174:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763479.003178:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763479.003182:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565878016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763479.066826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763480.090905:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763481.114869:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763482.138834:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763483.162857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763484.122721:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763484.122731:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763484.122736:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763484.122746:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3419 00000100:00080000:3.0:1789763484.122773:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763484.122782:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763484.122845:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565878144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763484.122869:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763484.122872:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763484.122874:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763484.122877:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3419 00000100:00080000:3.0:1789763484.122881:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763484.122884:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763484.122908:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763484.122920:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565878144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763484.122922:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565878272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763484.122936:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763484.122938:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763484.122941:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763484.122944:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3419 00000100:00080000:3.0:1789763484.122948:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763484.122951:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763484.122972:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763484.122985:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565878400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763484.122993:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763484.122996:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763484.122998:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763484.123001:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3419 00000100:00080000:3.0:1789763484.123004:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763484.123016:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1789763484.123024:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1789763484.123025:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763484.123039:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565878528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763484.123081:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763484/real 1789763484] req@ffff93f844d65c00 x1876699565878144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763484.123125:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565878144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763484.123134:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763484.123137:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763484.123140:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565878144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763484.123177:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565878272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763484.123209:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763484.123220:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763484/real 1789763484] req@ffff93f844d67480 x1876699565878272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763484.123230:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565878272/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763484.123238:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763484.123240:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763484.123253:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565878272:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763484.123258:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565878400:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763484.123275:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763484.123299:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763484/real 1789763484] req@ffff93f844d66a00 x1876699565878400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763484.123309:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565878400/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763484.123319:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763484.123323:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763484.123327:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565878400:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763484.123334:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565878528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763484.123388:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763484.123403:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763484/real 1789763484] req@ffff93f844d64e00 x1876699565878528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763484.123419:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565878528/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763484.123430:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763484.123434:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763484.123437:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565878528:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763484.186868:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763485.210870:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763486.234874:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763487.258822:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763488.282858:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763489.242776:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763489.242787:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763489.242792:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763489.242803:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3424 00000100:00080000:0.0:1789763489.242813:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763489.242821:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763489.242863:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565878656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763489.242894:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763489.242900:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763489.242903:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763489.242908:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3424 00000100:00080000:0.0:1789763489.242914:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763489.242920:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763489.242932:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763489.242950:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565878784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763489.242965:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565878656:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763489.242975:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763489.242979:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763489.242982:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763489.242987:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3424 00000100:00080000:0.0:1789763489.242992:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763489.242997:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763489.243003:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763489.243016:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565878912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763489.243030:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763489.243034:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763489.243037:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1789763489.243040:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763489.243042:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3424 00000100:00080000:0.0:1789763489.243046:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763489.243051:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763489.243057:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763489.243069:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565879040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763489.243090:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763489/real 1789763489] req@ffff93f850d43800 x1876699565878656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763489.243116:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565878656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763489.243133:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763489.243140:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763489.243145:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565878656:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763489.243173:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565878784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763489.243201:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763489.243215:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763489/real 1789763489] req@ffff93f850d41f80 x1876699565878784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763489.243229:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565878784/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763489.243241:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763489.243246:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763489.243251:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565878784:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763489.243259:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565878912:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763489.243290:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763489.243327:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763489/real 1789763489] req@ffff93f850d41500 x1876699565878912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763489.243342:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565878912/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763489.243374:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763489.243378:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763489.243381:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565878912:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763489.243388:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565879040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763489.243414:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763489.243425:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763489/real 1789763489] req@ffff93f850d43480 x1876699565879040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763489.243437:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565879040/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763489.243446:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763489.243449:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763489.243452:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565879040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763489.306822:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763490.330860:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763491.354807:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763492.378844:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763493.402847:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763494.363349:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763494.363364:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763494.363371:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763494.363386:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3429 00000100:00080000:1.0:1789763494.363400:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763494.363413:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763494.363465:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565879168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763494.363518:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763494.363532:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763494.363536:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763494.363543:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3429 00000100:00080000:1.0:1789763494.363551:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763494.363559:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763494.363575:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763494.363608:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565879296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763494.363609:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565879168:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763494.363680:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763494.363688:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763494.363692:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763494.363702:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3429 00000100:00080000:1.0:1789763494.363730:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763494.363739:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789763494.363747:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789763494.363755:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763494.363784:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565879424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763494.363803:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763494/real 1789763494] req@ffff93f844cc9c00 x1876699565879168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763494.363807:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763494.363824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763494.363832:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763494.363849:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3429 00000100:00080000:1.0:1789763494.363860:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789763494.363868:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565879168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763494.363873:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763494.363888:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00080000:0.0:1789763494.363898:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763494.363909:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763494.363918:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565879552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763494.363921:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565879168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763494.363975:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565879296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763494.364013:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763494.364031:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763494/real 1789763494] req@ffff93f844cca680 x1876699565879296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763494.364046:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565879296/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763494.364057:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763494.364060:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763494.364064:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565879296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763494.364071:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565879424:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763494.364091:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763494.364102:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763494/real 1789763494] req@ffff93f844cc8e00 x1876699565879424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763494.364130:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565879424/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763494.364147:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763494.364150:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763494.364154:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565879424:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763494.364160:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565879552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763494.364181:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763494.364193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763494/real 1789763494] req@ffff93f844ccb100 x1876699565879552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763494.364214:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565879552/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763494.364223:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763494.364227:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763494.364230:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565879552:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763494.426776:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763495.450866:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763496.474834:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763497.498781:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763498.522908:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763499.482733:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763499.482750:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763499.482757:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763499.482785:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3434 00000100:00080000:1.0:1789763499.482798:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763499.482807:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763499.482844:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565879680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763499.482872:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763499.482877:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763499.482880:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763499.482886:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3434 00000100:00080000:1.0:1789763499.482900:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763499.482907:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763499.482915:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763499.482945:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565879808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763499.482954:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763499.482958:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763499.482959:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763499.482965:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3434 00000100:00100000:2.0:1789763499.482966:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565879680:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763499.482969:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763499.482975:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763499.482979:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763499.482989:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565879936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763499.482999:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763499.483002:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763499.483004:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763499.483007:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3434 00000100:00080000:1.0:1789763499.483014:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763499.483018:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763499.483024:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789763499.483029:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763499.483037:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565880064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763499.483076:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763499/real 1789763499] req@ffff93f844ccad80 x1876699565879680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763499.483094:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565879680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763499.483103:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763499.483106:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763499.483110:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565879680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763499.483116:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565879808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763499.483131:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763499.483140:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763499/real 1789763499] req@ffff93f844cc9500 x1876699565879808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763499.483151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565879808/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763499.483157:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763499.483160:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763499.483164:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565879808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763499.483197:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565879936:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763499.483214:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763499.483222:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763499/real 1789763499] req@ffff93f844ccb100 x1876699565879936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763499.483244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565879936/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763499.483252:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763499.483254:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763499.483256:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565879936:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763499.483260:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565880064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763499.483275:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763499.483283:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763499/real 1789763499] req@ffff93f844cc8e00 x1876699565880064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763499.483290:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565880064/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763499.483296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763499.483298:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763499.483300:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565880064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763499.546794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763500.570877:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763501.594838:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763502.618903:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763503.642864:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763504.602849:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763504.602864:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763504.602870:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763504.602886:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3439 00000100:00080000:2.0:1789763504.602914:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763504.602928:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763504.603003:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565880192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763504.603067:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763504.603072:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763504.603075:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763504.603081:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3439 00000100:00080000:2.0:1789763504.603087:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763504.603098:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763504.603114:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763504.603136:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565880320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763504.603161:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763504.603166:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763504.603171:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789763504.603178:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565880192:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763504.603180:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3439 00000100:00080000:2.0:1789763504.603212:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763504.603237:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763504.603247:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763504.603269:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565880448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763504.603296:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763504.603302:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763504.603306:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763504.603313:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3439 00000100:00080000:2.0:1789763504.603323:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763504.603329:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763504.603336:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763504.603353:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565880576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763504.603370:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763504.603463:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763504/real 1789763504] req@ffff93f8517c2300 x1876699565880192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763504.603489:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565880192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763504.603505:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763504.603511:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763504.603530:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565880192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763504.603568:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565880320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763504.603599:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763504.603618:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763504/real 1789763504] req@ffff93f8517c1f80 x1876699565880320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763504.603635:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565880320/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763504.603705:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763504.603711:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763504.603716:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565880320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763504.603725:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565880448:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763504.603754:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763504.603771:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763504/real 1789763504] req@ffff93f8517c3b80 x1876699565880448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763504.603786:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565880448/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763504.603797:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763504.603802:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763504.603806:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565880448:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763504.603812:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565880576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763504.603833:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763504.603845:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763504/real 1789763504] req@ffff93f8517c0000 x1876699565880576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763504.603865:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565880576/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763504.603883:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763504.603890:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763504.603895:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565880576:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763504.666868:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763505.690933:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763506.714852:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763507.738882:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763508.762920:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763509.722802:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763509.722824:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763509.722842:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763509.722863:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3444 00000100:00080000:2.0:1789763509.722882:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763509.722897:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763509.722974:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565880704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763509.723036:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763509.723041:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763509.723044:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763509.723050:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3444 00000100:00080000:2.0:1789763509.723056:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763509.723062:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763509.723077:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763509.723108:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565880832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763509.723124:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763509.723128:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763509.723132:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763509.723137:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3444 00000100:00080000:2.0:1789763509.723141:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763509.723148:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763509.723182:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763509.723203:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565880704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763509.723218:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565880960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763509.723245:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763509.723254:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763509.723259:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763509.723269:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3444 00000100:00080000:2.0:1789763509.723280:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763509.723288:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763509.723299:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789763509.723324:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763509.723335:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565881088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763509.723405:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763509/real 1789763509] req@ffff93f8517c0000 x1876699565880704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763509.723443:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565880704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763509.723458:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763509.723463:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763509.723469:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565880704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763509.723479:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565880832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763509.723506:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763509.723522:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763509/real 1789763509] req@ffff93f8517c3b80 x1876699565880832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763509.723537:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565880832/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763509.723566:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763509.723570:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763509.723575:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565880832:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763509.723619:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565880960:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763509.723741:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763509.723761:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763509/real 1789763509] req@ffff93f8517c1f80 x1876699565880960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763509.723777:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565880960/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763509.723788:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763509.723792:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763509.723796:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565880960:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763509.723804:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565881088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763509.723828:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763509.723845:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763509/real 1789763509] req@ffff93f8517c2300 x1876699565881088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763509.723860:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565881088/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763509.723869:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763509.723872:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763509.723877:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565881088:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763509.786876:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763510.810857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763511.834876:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763512.858888:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763513.882895:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763514.842825:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763514.842849:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763514.842859:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763514.842893:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3449 00000100:00080000:2.0:1789763514.842936:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763514.842953:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763514.843022:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565881216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763514.843075:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763514.843080:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763514.843083:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763514.843088:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3449 00000100:00080000:2.0:1789763514.843094:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763514.843099:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763514.843136:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763514.843165:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565881344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763514.843182:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565881216:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763514.843190:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763514.843197:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763514.843200:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763514.843209:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3449 00000100:00080000:2.0:1789763514.843217:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763514.843228:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763514.843241:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763514.843274:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565881472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763514.843297:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763514.843305:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763514.843309:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763514.843318:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3449 00000100:00080000:2.0:1789763514.843329:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763514.843341:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763514.843353:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763514.843373:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565881344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763514.843386:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565881600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763514.843450:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565881472:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763514.843494:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565881600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763514.906807:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763515.930875:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763516.954883:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789763517.915082:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789763517.915202:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789763517.915236:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763517.915315:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763517.915342:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763517.915371:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789763517.915403:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763514/real 1789763517] req@ffff93f8517c1500 x1876699565881344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763517.915462:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565881344/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763517.915484:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763517.915493:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763517.915501:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565881344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763517.915515:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763514/real 1789763517] req@ffff93f8517c2680 x1876699565881472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763517.915552:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565881472/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763517.915568:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763517.915575:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763517.915582:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565881472:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763517.915597:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763514/real 1789763517] req@ffff93f8517c0380 x1876699565881216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763517.915621:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565881216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763517.915637:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763517.915733:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763517.915742:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565881216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763517.915789:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763514/real 1789763517] req@ffff93f8517c0a80 x1876699565881600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763517.915824:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565881600/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763517.915844:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763517.915852:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763517.915859:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565881600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763517.978856:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763519.002935:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763519.962807:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763519.962839:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763519.962847:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763519.962867:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3454 00000100:00080000:2.0:1789763519.962890:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763519.962906:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763519.962988:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565881728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763519.963064:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763519.963069:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763519.963072:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763519.963115:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3454 00000100:00080000:2.0:1789763519.963123:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763519.963132:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763519.963151:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763519.963178:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565881856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763519.963198:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763519.963203:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763519.963207:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763519.963215:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3454 00000100:00080000:2.0:1789763519.963222:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763519.963229:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763519.963239:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763519.963259:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565881984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763519.963289:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763519.963295:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763519.963298:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763519.963305:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3454 00000100:00080000:2.0:1789763519.963322:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763519.963329:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763519.963337:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763519.963356:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565882112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763519.963462:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565881728:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1789763519.963567:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763519.963635:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763519/real 1789763519] req@ffff93f8517c2d80 x1876699565881728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763519.963709:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565881728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763519.963896:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763519.963905:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763519.963911:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565881728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763519.963923:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565881856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763519.963967:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763519.963993:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763519/real 1789763519] req@ffff93f8517c2a00 x1876699565881856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763519.964036:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565881856/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763519.964053:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763519.964060:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763519.964067:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565881856:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763519.964103:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565881984:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763519.964142:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763519.964164:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763519/real 1789763519] req@ffff93f8517c1180 x1876699565881984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763519.964183:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565881984/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763519.964198:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763519.964204:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763519.964209:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565881984:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763519.964221:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565882112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763519.964255:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763519.964277:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763519/real 1789763519] req@ffff93f8517c3100 x1876699565882112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763519.964298:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565882112/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763519.964313:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763519.964319:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763519.964325:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565882112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763520.026878:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763521.050860:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763522.074967:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763523.098947:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763524.122832:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763525.082900:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763525.082920:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763525.082930:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763525.082950:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3460 00000100:00080000:2.0:1789763525.082973:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763525.082989:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763525.083132:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565882240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763525.083185:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763525.083193:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763525.083197:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763525.083205:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3460 00000100:00080000:2.0:1789763525.083214:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763525.083224:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763525.083241:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763525.083271:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565882240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763525.083292:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565882368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763525.083321:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763525.083331:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763525.083335:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763525.083361:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3460 00000100:00080000:2.0:1789763525.083370:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789763525.083377:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763525.083388:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763525.083404:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763525.083438:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565882496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763525.083445:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763525/real 1789763525] req@ffff93f8517c1880 x1876699565882240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763525.083536:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789763525.083539:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565882240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763525.083565:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763525.083577:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763525.083584:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763525.083596:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3460 00000100:00080000:0.0:1789763525.083609:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:2.0:1789763525.083610:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789763525.083623:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565882240:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763525.083633:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763525.083649:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763525.083705:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565882368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763525.083740:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763525.083742:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565882624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763525.083761:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763525/real 1789763525] req@ffff93f8517c3480 x1876699565882368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763525.083815:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565882368/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763525.083845:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763525.083853:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763525.083861:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565882368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763525.083875:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565882496:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763525.083916:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763525.083934:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763525/real 1789763525] req@ffff93f8517c3800 x1876699565882496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763525.083949:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565882496/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763525.083974:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763525.083996:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763525.084000:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565882496:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763525.084022:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565882624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763525.084076:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763525.084099:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763525/real 1789763525] req@ffff93f8517c0e00 x1876699565882624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763525.084124:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565882624/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763525.084140:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763525.084146:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763525.084150:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565882624:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763525.146987:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763526.170859:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763527.194871:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763528.218960:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763529.242834:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763530.202758:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763530.202767:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763530.202771:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763530.202779:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3465 00000100:00080000:0.0:1789763530.202788:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763530.202794:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763530.202829:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565882752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763530.202862:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763530.202884:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763530.202899:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763530.202908:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3465 00000100:00080000:0.0:1789763530.202917:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763530.202925:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763530.202936:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565882752:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1789763530.202942:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763530.202975:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565882880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763530.202993:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763530.202997:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763530.203000:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763530.203036:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3465 00000100:00080000:0.0:1789763530.203044:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789763530.203047:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763530.203072:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763530.203078:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763530.203100:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565883008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763530.203109:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763530.203112:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763530.203114:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763530.203120:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3465 00000100:00100000:2.0:1789763530.203121:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763530/real 1789763530] req@ffff93f850d43100 x1876699565882752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763530.203124:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763530.203132:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763530.203140:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763530.203155:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565883136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763530.203161:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565882752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763530.203175:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763530.203179:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763530.203183:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565882752:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763530.203205:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565882880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763530.203223:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763530.203242:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763530/real 1789763530] req@ffff93f850d41180 x1876699565882880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763530.203252:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565882880/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763530.203258:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763530.203261:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763530.203263:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565882880:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763530.203267:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565883008:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763530.203281:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763530.203288:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763530/real 1789763530] req@ffff93f850d41c00 x1876699565883008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763530.203296:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565883008/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763530.203307:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763530.203309:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763530.203312:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565883008:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763530.203315:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565883136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763530.203326:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763530.203332:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763530/real 1789763530] req@ffff93f850d42680 x1876699565883136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763530.203340:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565883136/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763530.203346:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763530.203355:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763530.203357:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565883136:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763530.266758:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763531.290898:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763532.314855:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763533.338817:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763534.362837:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763535.322803:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763535.322816:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763535.322824:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763535.322838:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3470 00000100:00080000:0.0:1789763535.322853:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763535.322865:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763535.322914:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565883264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763535.322954:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763535.322959:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763535.322987:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763535.322995:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3470 00000100:00080000:0.0:1789763535.323002:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763535.323009:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763535.323024:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763535.323047:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565883264:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763535.323052:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565883392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763535.323076:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763535.323081:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763535.323084:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763535.323090:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3470 00000100:00080000:0.0:1789763535.323095:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763535.323102:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763535.323109:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763535.323126:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565883520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763535.323142:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763535.323143:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763535.323153:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763535.323156:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763535.323165:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3470 00000100:00080000:0.0:1789763535.323183:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763535.323189:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763535.323196:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763535.323203:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763535/real 1789763535] req@ffff93f850d40e00 x1876699565883264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763535.323218:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565883648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763535.323240:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565883264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763535.323257:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763535.323266:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763535.323275:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565883264:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763535.323358:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565883392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763535.323402:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763535.323433:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763535/real 1789763535] req@ffff93f850d40000 x1876699565883392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763535.323450:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565883392/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763535.323461:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763535.323466:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763535.323470:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565883392:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763535.323477:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565883520:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763535.323504:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763535.323526:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763535/real 1789763535] req@ffff93f850d42300 x1876699565883520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763535.323542:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565883520/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763535.323553:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763535.323557:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763535.323561:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565883520:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763535.323569:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565883648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763535.323595:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763535.323611:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763535/real 1789763535] req@ffff93f850d40380 x1876699565883648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763535.323627:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565883648/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763535.323637:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763535.323721:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763535.323733:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565883648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763535.386850:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763536.410873:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763537.434879:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763538.458874:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763539.482810:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763540.442784:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763540.442793:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763540.442796:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763540.442804:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3475 00000100:00080000:3.0:1789763540.442811:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763540.442818:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763540.442849:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565883776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763540.442870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763540.442873:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763540.442874:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763540.442877:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3475 00000100:00080000:3.0:1789763540.442879:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763540.442882:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763540.442890:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763540.442898:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565883904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763540.442906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763540.442908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763540.442910:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789763540.442914:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565883776:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763540.442916:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3475 00000100:00080000:3.0:1789763540.442920:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763540.442925:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763540.442940:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763540.442951:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565884032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763540.442958:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763540.442961:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763540.442963:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763540.442966:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3475 00000100:00080000:3.0:1789763540.442976:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763540.442979:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763540.442984:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789763540.442988:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763540.442996:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565884160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763540.443043:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763540/real 1789763540] req@ffff93f844d67100 x1876699565883776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763540.443063:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565883776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763540.443077:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763540.443083:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763540.443088:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565883776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763540.443112:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565883904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763540.443131:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763540.443140:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763540/real 1789763540] req@ffff93f844d67800 x1876699565883904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763540.443148:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565883904/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763540.443159:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763540.443161:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763540.443163:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565883904:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763540.443167:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565884032:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763540.443179:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763540.443185:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763540/real 1789763540] req@ffff93f844d64700 x1876699565884032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763540.443193:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565884032/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763540.443198:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763540.443200:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763540.443203:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565884032:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763540.443209:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565884160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763540.443227:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763540.443239:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763540/real 1789763540] req@ffff93f844d67b80 x1876699565884160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763540.443253:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565884160/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763540.443259:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763540.443261:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763540.443264:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565884160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763540.506792:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763541.530903:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763542.554844:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763543.578842:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763544.602845:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763545.562829:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763545.562846:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763545.562854:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763545.562870:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3480 00000100:00080000:1.0:1789763545.562887:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763545.562928:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763545.562988:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565884288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763545.563037:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763545.563046:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763545.563051:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763545.563062:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3480 00000100:00080000:1.0:1789763545.563071:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763545.563088:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763545.563098:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565884288:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789763545.563124:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763545.563160:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565884416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763545.563183:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763545.563190:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763545.563194:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763545.563202:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3480 00000800:00000100:2.0:1789763545.563210:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763545.563217:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763545.563231:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763545.563243:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763545.563269:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565884544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763545.563289:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763545/real 1789763545] req@ffff93f844cca680 x1876699565884288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763545.563306:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763545.563322:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763545.563327:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789763545.563338:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565884288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763545.563340:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3480 00000100:00080000:1.0:1789763545.563355:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763545.563376:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763545.563387:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763545.563412:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565884672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763545.563414:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763545.563431:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763545.563445:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565884288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763545.563506:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565884416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763545.563559:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763545.563596:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763545/real 1789763545] req@ffff93f844cc9c00 x1876699565884416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763545.563620:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565884416/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763545.563701:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763545.563711:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763545.563720:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565884416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763545.563732:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565884544:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763545.563768:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763545.563786:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763545/real 1789763545] req@ffff93f844cc8700 x1876699565884544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763545.563802:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565884544/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763545.563813:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763545.563817:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763545.563821:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565884544:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763545.563828:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565884672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763545.563853:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763545.563874:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763545/real 1789763545] req@ffff93f844ccaa00 x1876699565884672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763545.563929:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565884672/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763545.563957:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763545.563963:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763545.563971:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565884672:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763545.626935:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763546.650838:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763547.674945:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763548.698757:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763549.722738:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763550.682807:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763550.682821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763550.682828:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763550.682843:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3485 00000100:00080000:1.0:1789763550.682882:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763550.682894:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763550.682956:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565884800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763550.682990:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763550.682995:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763550.682998:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763550.683005:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3485 00000100:00080000:1.0:1789763550.683011:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763550.683017:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763550.683029:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763550.683044:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565884800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763550.683047:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565884928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763550.683087:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763550.683091:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763550.683094:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763550.683099:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3485 00000100:00080000:1.0:1789763550.683106:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763550.683110:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763550.683117:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763550.683152:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565885056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763550.683183:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763550.683187:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763550.683191:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763550.683193:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763550.683200:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3485 00000100:00080000:1.0:1789763550.683207:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763550.683212:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763550.683218:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763550.683235:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565885184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763550.683244:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763550/real 1789763550] req@ffff93f844ccb480 x1876699565884800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763550.683268:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565884800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763550.683280:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763550.683285:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763550.683289:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565884800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763550.683317:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565884928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763550.683341:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763550.683354:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763550/real 1789763550] req@ffff93f844ccb800 x1876699565884928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763550.683366:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565884928/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763550.683375:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763550.683378:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763550.683381:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565884928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763550.683387:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565885056:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763550.683406:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763550.683415:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763550/real 1789763550] req@ffff93f844ccbb80 x1876699565885056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763550.683429:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565885056/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763550.683443:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763550.683448:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763550.683452:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565885056:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763550.683462:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565885184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763550.683506:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763550.683525:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763550/real 1789763550] req@ffff93f844cc8000 x1876699565885184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763550.683539:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565885184/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763550.683547:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763550.683550:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763550.683553:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565885184:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763550.746933:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763551.770921:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763552.794884:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763553.818928:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763554.842899:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763555.802855:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763555.802926:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763555.802932:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763555.802945:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3490 00000100:00080000:1.0:1789763555.802957:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763555.802967:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763555.803013:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565885312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763555.803052:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763555.803057:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763555.803059:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763555.803080:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3490 00000100:00080000:1.0:1789763555.803085:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763555.803091:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763555.803105:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763555.803122:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565885440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763555.803131:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565885312:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763555.803149:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763555.803155:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763555.803158:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763555.803165:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3490 00000100:00080000:1.0:1789763555.803169:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763555.803173:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763555.803179:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763555.803195:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565885568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763555.803208:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763555.803213:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763555.803216:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763555.803220:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3490 00000800:00000100:0.0:1789763555.803231:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763555.803241:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763555.803251:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763555.803258:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763555.803275:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565885696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763555.803298:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763555/real 1789763555] req@ffff93f844cc8380 x1876699565885312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763555.803352:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565885312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763555.803367:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763555.803372:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763555.803378:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565885312:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763555.803414:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565885440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763555.803462:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763555.803491:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763555/real 1789763555] req@ffff93f844cca300 x1876699565885440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763555.803513:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565885440/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763555.803525:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763555.803529:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763555.803534:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565885440:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763555.803542:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565885568:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763555.803572:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763555.803586:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763555/real 1789763555] req@ffff93f844cc9f80 x1876699565885568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763555.803601:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565885568/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763555.803620:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763555.803624:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763555.803629:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565885568:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763555.803635:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565885696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763555.803702:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763555.803718:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763555/real 1789763555] req@ffff93f844cc9880 x1876699565885696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763555.803748:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565885696/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763555.803759:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763555.803763:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763555.803767:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565885696:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763555.866936:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763556.890888:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763557.914794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763558.938911:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763559.962865:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763560.922814:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763560.922830:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763560.922839:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763560.922863:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3495 00000100:00080000:1.0:1789763560.922879:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763560.922892:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763560.922949:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565885824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763560.923003:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763560.923011:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763560.923015:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763560.923024:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3495 00000100:00080000:1.0:1789763560.923033:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763560.923042:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763560.923059:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763560.923085:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565885952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763560.923103:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763560.923109:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763560.923113:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763560.923121:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3495 00000100:00080000:1.0:1789763560.923128:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763560.923134:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763560.923143:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763560.923166:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565886080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763560.923223:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763560.923232:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763560.923236:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763560.923247:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3495 00000100:00080000:1.0:1789763560.923256:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763560.923264:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763560.923275:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763560.923308:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565886208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763560.923327:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565885824:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1789763560.923521:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763560.923623:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763560/real 1789763560] req@ffff93f844cc8a80 x1876699565885824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763560.923721:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565885824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763560.923745:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763560.923755:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763560.923765:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565885824:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763560.923830:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565885952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763560.923896:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763560.923927:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763560/real 1789763560] req@ffff93f844cc9180 x1876699565885952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763560.923956:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565885952/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763560.923974:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763560.923981:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763560.923988:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565885952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763560.924011:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565886080:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763560.924056:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763560.924084:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763560/real 1789763560] req@ffff93f844cc9880 x1876699565886080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763560.924110:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565886080/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763560.924128:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763560.924135:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763560.924143:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565886080:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763560.924201:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565886208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763560.924249:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763560.924292:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763560/real 1789763560] req@ffff93f844cc9f80 x1876699565886208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763560.924310:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565886208/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763560.924322:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763560.924327:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763560.924332:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565886208:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763560.986890:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763562.010896:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763563.034941:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763564.058876:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763565.082894:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763566.042848:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763566.042863:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763566.042869:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763566.042884:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3501 00000100:00080000:1.0:1789763566.042899:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763566.042909:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763566.042952:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565886336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763566.042983:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763566.042987:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763566.042989:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763566.042994:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3501 00000100:00080000:1.0:1789763566.042998:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763566.043002:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763566.043014:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763566.043028:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565886464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763566.043047:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565886336:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763566.043050:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763566.043065:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763566.043068:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763566.043077:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3501 00000100:00080000:1.0:1789763566.043093:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763566.043101:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763566.043111:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763566.043135:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565886592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763566.043150:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763566.043164:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763566.043169:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763566.043172:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763566.043179:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3501 00000100:00080000:1.0:1789763566.043185:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763566.043190:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763566.043196:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763566.043208:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763566/real 1789763566] req@ffff93f844cca300 x1876699565886336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763566.043215:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565886720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763566.043268:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565886336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763566.043289:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763566.043298:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763566.043306:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565886336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763566.043338:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565886464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763566.043369:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763566.043385:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763566/real 1789763566] req@ffff93f844cc8380 x1876699565886464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763566.043400:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565886464/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763566.043418:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763566.043422:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763566.043426:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565886464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763566.043432:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565886592:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763566.043470:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763566.043482:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763566/real 1789763566] req@ffff93f844cc8000 x1876699565886592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763566.043497:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565886592/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763566.043513:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763566.043520:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763566.043526:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565886592:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763566.043537:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565886720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763566.043575:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763566.043598:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763566/real 1789763566] req@ffff93f844ccbb80 x1876699565886720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763566.043624:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565886720/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763566.043634:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763566.043638:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763566.043642:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565886720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763566.106922:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763567.130864:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763568.154910:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763569.178849:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763570.202908:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763571.162840:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763571.162855:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763571.162860:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763571.162875:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3506 00000100:00080000:3.0:1789763571.162891:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763571.162901:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763571.162961:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565886848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763571.163002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763571.163007:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763571.163008:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763571.163013:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3506 00000100:00080000:3.0:1789763571.163016:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763571.163020:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763571.163032:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763571.163054:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565886976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763571.163067:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763571.163070:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763571.163073:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763571.163080:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3506 00000100:00080000:3.0:1789763571.163088:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1789763571.163093:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565886848:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763571.163094:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763571.163104:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763571.163131:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565887104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763571.163143:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763571.163147:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763571.163149:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763571.163154:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3506 00000100:00080000:3.0:1789763571.163159:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763571.163165:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763571.163173:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763571.163193:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565887232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1789763571.163219:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763571.163286:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763571/real 1789763571] req@ffff93f844d65880 x1876699565886848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763571.163322:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565886848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763571.163344:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763571.163354:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763571.163363:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565886848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763571.163409:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565886976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763571.163447:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763571.163470:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763571/real 1789763571] req@ffff93f844d66680 x1876699565886976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763571.163487:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565886976/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763571.163500:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763571.163504:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763571.163509:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565886976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763571.163518:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565887104:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763571.163541:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763571.163555:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763571/real 1789763571] req@ffff93f844d67b80 x1876699565887104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763571.163576:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565887104/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763571.163596:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763571.163601:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763571.163607:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565887104:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763571.163616:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565887232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763571.163689:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763571.163744:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763571/real 1789763571] req@ffff93f844d64700 x1876699565887232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763571.163779:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565887232/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763571.163800:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763571.163808:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763571.163816:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565887232:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763571.226867:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763572.250877:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763573.274893:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763574.298920:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763575.322953:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763576.282914:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763576.282933:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763576.282943:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763576.282964:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3511 00000100:00080000:1.0:1789763576.282992:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763576.283006:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763576.283087:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565887360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763576.283144:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763576.283153:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763576.283157:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763576.283166:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3511 00000100:00080000:1.0:1789763576.283174:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763576.283192:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763576.283209:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763576.283234:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565887488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763576.283254:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763576.283261:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763576.283264:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763576.283273:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3511 00000100:00080000:1.0:1789763576.283281:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789763576.283289:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565887360:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763576.283295:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763576.283312:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763576.283340:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565887616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763576.283383:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763576.283390:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763576.283394:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763576.283403:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3511 00000100:00080000:1.0:1789763576.283411:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789763576.283423:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763576.283425:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763576.283461:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763576.283492:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565887744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763576.283543:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763576/real 1789763576] req@ffff93f844ccb800 x1876699565887360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763576.283585:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565887360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763576.283618:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763576.283627:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763576.283636:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565887360:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763576.283715:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565887488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763576.283764:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763576.283786:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763576/real 1789763576] req@ffff93f844ccb480 x1876699565887488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763576.283803:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565887488/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763576.283814:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763576.283818:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763576.283822:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565887488:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763576.283856:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565887616:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763576.283880:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763576.283894:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763576/real 1789763576] req@ffff93f844ccaa00 x1876699565887616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763576.283909:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565887616/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763576.283919:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763576.283923:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763576.283927:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565887616:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763576.283933:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565887744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763576.283952:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763576.283963:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763576/real 1789763576] req@ffff93f844cc8700 x1876699565887744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763576.283986:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565887744/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763576.283996:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763576.283999:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763576.284003:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565887744:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763576.346850:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763577.370906:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763578.394881:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763579.418841:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763580.442783:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763581.402829:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763581.402841:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763581.402847:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763581.402860:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3516 00000100:00080000:1.0:1789763581.402872:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763581.402883:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763581.402937:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565887872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763581.402982:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763581.402989:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763581.402994:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763581.403005:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3516 00000100:00080000:1.0:1789763581.403014:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763581.403024:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763581.403032:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565887872:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789763581.403044:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763581.403081:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565888000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763581.403098:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763581.403103:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763581.403106:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763581.403111:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3516 00000100:00080000:1.0:1789763581.403118:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763581.403124:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763581.403131:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763581.403146:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565888128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763581.403160:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763581.403165:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763581.403168:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763581.403175:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3516 00000100:00080000:1.0:1789763581.403179:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763581.403186:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763581.403194:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565888000:192.168.203.147@tcp:38:kworker.0 00000008:00100000:1.0:1789763581.403196:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763581.403227:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565888256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763581.403263:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565888128:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763581.403322:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565888256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763581.466888:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763582.490869:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763583.514814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789763584.475062:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789763584.475126:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789763584.475146:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763584.475197:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763584.475208:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763584.475218:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789763584.475300:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763581/real 1789763584] req@ffff93f844ccb100 x1876699565888256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763584.475362:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565888256/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763584.475381:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763584.475389:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763584.475395:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565888256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763584.475408:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763581/real 1789763584] req@ffff93f844cca680 x1876699565888000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763584.475438:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565888000/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763584.475449:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763584.475454:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763584.475459:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565888000:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763584.475470:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763581/real 1789763584] req@ffff93f844cc8e00 x1876699565888128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763584.475486:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565888128/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763584.475497:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763584.475501:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763584.475506:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565888128:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763584.475517:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763581/real 1789763584] req@ffff93f844cc9c00 x1876699565887872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763584.475533:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565887872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763584.475544:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763584.475548:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763584.475553:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565887872:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789763584.538902:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763585.562808:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763586.522757:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763586.522770:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763586.522794:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763586.522808:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3521 00000100:00080000:1.0:1789763586.522824:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763586.522836:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763586.522887:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565888384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763586.522926:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763586.522934:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763586.522949:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763586.522956:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3521 00000100:00080000:1.0:1789763586.522962:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763586.522986:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763586.522998:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763586.523002:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565888384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763586.523020:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565888512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763586.523042:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763586.523049:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763586.523053:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763586.523060:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3521 00000100:00080000:1.0:1789763586.523086:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763586.523094:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1789763586.523104:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1789763586.523121:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763586.523154:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565888640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763586.523172:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763586/real 1789763586] req@ffff93f844cc9500 x1876699565888384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763586.523177:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763586.523193:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763586.523200:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763586.523214:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3521 00000100:00080000:1.0:1789763586.523224:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1789763586.523233:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565888384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763586.523236:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763586.523253:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00080000:0.0:1789763586.523266:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763586.523278:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763586.523287:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565888768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763586.523295:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565888384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763586.523352:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565888512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763586.523400:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763586.523426:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763586/real 1789763586] req@ffff93f844ccad80 x1876699565888512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763586.523450:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565888512/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763586.523468:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763586.523474:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763586.523479:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565888512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763586.523490:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565888640:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763586.523525:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763586.523544:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763586/real 1789763586] req@ffff93f844cc9c00 x1876699565888640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763586.523563:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565888640/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763586.523576:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763586.523581:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763586.523611:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565888640:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763586.523623:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565888768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763586.523716:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763586.523740:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763586/real 1789763586] req@ffff93f844cc8e00 x1876699565888768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763586.523763:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565888768/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763586.523778:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763586.523783:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763586.523790:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565888768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763586.586823:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763587.610881:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763588.634800:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763589.658751:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763590.682779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763591.642765:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763591.642781:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763591.642787:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763591.642802:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3526 00000100:00080000:1.0:1789763591.642826:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763591.642838:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763591.642903:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565888896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763591.642936:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763591.642940:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763591.642954:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763591.642960:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3526 00000100:00080000:1.0:1789763591.642966:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763591.642972:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763591.642983:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763591.642985:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565888896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763591.643005:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565889024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763591.643020:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763591.643025:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763591.643027:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763591.643033:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3526 00000100:00080000:1.0:1789763591.643041:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763591.643049:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763591.643060:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789763591.643080:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763591.643086:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565889152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763591.643110:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763591.643116:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763591.643120:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763591.643128:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3526 00000100:00100000:0.0:1789763591.643134:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763591/real 1789763591] req@ffff93f844cca680 x1876699565888896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763591.643138:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763591.643151:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763591.643165:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763591.643166:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565888896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763591.643186:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763591.643192:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763591.643199:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565889280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763591.643202:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565888896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763591.643244:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565889024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763591.643288:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763591.643313:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763591/real 1789763591] req@ffff93f844ccb100 x1876699565889024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763591.643331:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565889024/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763591.643341:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763591.643345:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763591.643349:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565889024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763591.643356:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565889152:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763591.643381:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763591.643394:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763591/real 1789763591] req@ffff93f844cc8700 x1876699565889152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763591.643407:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565889152/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763591.643416:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763591.643419:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763591.643422:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565889152:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763591.643428:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565889280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763591.643445:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763591.643455:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763591/real 1789763591] req@ffff93f844ccaa00 x1876699565889280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763591.643467:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565889280/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763591.643476:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763591.643479:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763591.643482:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565889280:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763591.706785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763592.730805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763593.754804:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763594.778912:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763595.802911:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763596.762818:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763596.762830:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763596.762837:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763596.762866:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3531 00000100:00080000:3.0:1789763596.762882:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763596.762894:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763596.762960:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565889408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763596.763011:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763596.763016:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763596.763019:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763596.763024:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3531 00000100:00080000:3.0:1789763596.763032:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763596.763039:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763596.763051:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763596.763072:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565889536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763596.763087:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565889408:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763596.763094:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763596.763113:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763596.763116:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763596.763124:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3531 00000100:00080000:3.0:1789763596.763128:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763596.763132:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763596.763140:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763596.763157:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565889664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763596.763171:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763596.763177:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763596.763180:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763596.763185:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3531 00000100:00080000:3.0:1789763596.763191:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763596.763196:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763596.763204:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763596.763230:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565889792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763596.763232:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763596.763330:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763596/real 1789763596] req@ffff93f844d67800 x1876699565889408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763596.763371:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565889408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763596.763396:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763596.763406:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763596.763415:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565889408:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763596.763469:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565889536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763596.763559:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763596.763583:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763596/real 1789763596] req@ffff93f844d67100 x1876699565889536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763596.763600:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565889536/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763596.763612:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763596.763616:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763596.763621:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565889536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763596.763630:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565889664:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763596.763711:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763596.763728:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763596/real 1789763596] req@ffff93f844d64e00 x1876699565889664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763596.763744:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565889664/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763596.763756:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763596.763760:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763596.763764:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565889664:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763596.763773:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565889792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763596.763797:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763596.763810:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763596/real 1789763596] req@ffff93f844d66a00 x1876699565889792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763596.763825:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565889792/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763596.763834:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763596.763838:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763596.763841:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565889792:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763596.826857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763597.850902:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763598.874860:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763599.898850:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763600.922904:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763601.882931:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763601.882945:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763601.882950:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763601.882966:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3536 00000100:00080000:1.0:1789763601.882984:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763601.882995:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763601.883061:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565889920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763601.883118:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763601.883127:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763601.883130:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763601.883135:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3536 00000100:00080000:1.0:1789763601.883141:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763601.883146:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763601.883159:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763601.883173:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565890048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763601.883187:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763601.883192:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763601.883194:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763601.883200:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3536 00000100:00080000:1.0:1789763601.883203:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763601.883206:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763601.883213:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763601.883224:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565890176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763601.883237:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763601.883241:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763601.883243:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763601.883250:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3536 00000100:00080000:1.0:1789763601.883256:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763601.883266:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763601.883267:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565889920:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789763601.883275:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763601.883298:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565890304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763601.883394:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763601.883502:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763601/real 1789763601] req@ffff93f844ccb480 x1876699565889920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763601.883529:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565889920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763601.883547:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763601.883554:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763601.883562:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565889920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763601.883572:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565890048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763601.883605:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763601.883622:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763601/real 1789763601] req@ffff93f844ccb800 x1876699565890048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763601.883637:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565890048/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763601.883692:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763601.883697:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763601.883701:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565890048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763601.883735:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565890176:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763601.883762:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763601.883778:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763601/real 1789763601] req@ffff93f844ccbb80 x1876699565890176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763601.883793:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565890176/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763601.883803:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763601.883807:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763601.883811:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565890176:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763601.883819:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565890304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763601.883840:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763601.883851:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763601/real 1789763601] req@ffff93f844cc8000 x1876699565890304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763601.883865:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565890304/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763601.883875:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763601.883878:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763601.883882:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565890304:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763601.946876:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763602.970903:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763603.994835:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763605.018869:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763606.042879:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763607.002785:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763607.002808:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763607.002841:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763607.002862:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3541 00000100:00080000:1.0:1789763607.002883:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763607.002900:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763607.002966:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565890432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763607.003017:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763607.003023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763607.003027:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763607.003033:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3541 00000100:00080000:1.0:1789763607.003041:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763607.003049:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763607.003065:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763607.003082:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565890432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763607.003089:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565890560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763607.003108:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763607.003112:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763607.003115:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763607.003121:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3541 00000100:00080000:1.0:1789763607.003128:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763607.003133:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763607.003139:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763607.003156:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565890688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763607.003178:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763607.003186:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763607.003190:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763607.003200:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3541 00000800:00000100:0.0:1789763607.003211:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763607.003215:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763607.003230:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763607.003243:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763607.003270:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565890816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763607.003311:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763607/real 1789763607] req@ffff93f844cc8380 x1876699565890432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763607.003349:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565890432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763607.003365:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763607.003371:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763607.003377:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565890432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763607.003419:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565890560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763607.003506:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763607.003530:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763607/real 1789763607] req@ffff93f844cca300 x1876699565890560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763607.003551:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565890560/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763607.003563:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763607.003568:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763607.003572:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565890560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763607.003581:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565890688:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763607.003628:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763607.003717:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763607/real 1789763607] req@ffff93f844cc9f80 x1876699565890688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763607.003750:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565890688/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763607.003767:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763607.003774:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763607.003781:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565890688:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763607.003792:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565890816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763607.003840:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763607.003870:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763607/real 1789763607] req@ffff93f844cc9880 x1876699565890816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763607.003894:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565890816/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763607.003925:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763607.003934:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763607.003943:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565890816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763607.066824:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763608.090883:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763609.114870:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763610.138934:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763611.162882:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763612.122816:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763612.122832:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763612.122840:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763612.122857:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3547 00000100:00080000:3.0:1789763612.122891:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763612.122906:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763612.122978:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565890944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763612.123030:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763612.123036:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763612.123038:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763612.123045:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3547 00000100:00080000:3.0:1789763612.123051:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763612.123059:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763612.123074:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763612.123086:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565890944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763612.123099:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565891072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763612.123123:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763612.123128:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763612.123130:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763612.123135:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3547 00000100:00080000:3.0:1789763612.123141:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763612.123145:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763612.123152:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763612.123171:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565891200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763612.123190:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1789763612.123200:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763612.123203:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763612.123210:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763612.123220:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3547 00000100:00080000:3.0:1789763612.123231:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763612.123240:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763612.123250:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763612.123272:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763612/real 1789763612] req@ffff93f844d67480 x1876699565890944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763612.123283:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565891328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763612.123314:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565890944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763612.123332:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763612.123337:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763612.123345:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565890944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763612.123381:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565891072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763612.123454:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763612.123474:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763612/real 1789763612] req@ffff93f844d65c00 x1876699565891072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763612.123491:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565891072/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763612.123503:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763612.123507:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763612.123511:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565891072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763612.123520:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565891200:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763612.123545:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763612.123557:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763612/real 1789763612] req@ffff93f844d66d80 x1876699565891200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763612.123572:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565891200/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763612.123583:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763612.123586:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763612.123590:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565891200:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763612.123596:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565891328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763612.123617:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763612.123628:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763612/real 1789763612] req@ffff93f844d65500 x1876699565891328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763612.123642:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565891328/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763612.123652:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763612.123656:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763612.123690:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565891328:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763612.186886:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763613.210861:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763614.234844:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763615.258954:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763616.282871:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763617.242811:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763617.242832:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763617.242842:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763617.242864:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3552 00000100:00080000:0.0:1789763617.242883:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763617.242899:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763617.242972:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565891456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763617.243026:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763617.243035:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763617.243038:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763617.243044:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3552 00000100:00080000:0.0:1789763617.243051:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763617.243058:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763617.243073:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763617.243102:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565891584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763617.243105:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565891456:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763617.243133:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763617.243139:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763617.243141:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763617.243147:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3552 00000100:00080000:0.0:1789763617.243152:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763617.243156:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763617.243163:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763617.243180:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565891712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763617.243203:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763617.243208:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763617.243211:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763617.243216:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3552 00000100:00080000:0.0:1789763617.243221:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763617.243225:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763617.243231:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789763617.243233:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763617.243254:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565891840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763617.243309:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763617/real 1789763617] req@ffff93f850d43b80 x1876699565891456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763617.243336:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565891456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763617.243352:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763617.243358:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763617.243391:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565891456:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763617.243429:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565891584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763617.243464:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763617.243482:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763617/real 1789763617] req@ffff93f850d40700 x1876699565891584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763617.243500:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565891584/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763617.243511:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763617.243516:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763617.243520:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565891584:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763617.243527:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565891712:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763617.243549:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763617.243561:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763617/real 1789763617] req@ffff93f850d40380 x1876699565891712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763617.243576:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565891712/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763617.243586:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763617.243589:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763617.243593:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565891712:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763617.243600:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565891840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763617.243621:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763617.243632:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763617/real 1789763617] req@ffff93f850d42300 x1876699565891840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763617.243681:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565891840/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763617.243701:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763617.243706:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763617.243710:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565891840:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763617.306858:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763618.330840:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763619.354925:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763620.378882:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763621.402851:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763622.362810:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763622.362833:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763622.362842:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763622.362863:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3557 00000100:00080000:1.0:1789763622.362885:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763622.362915:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763622.362986:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565891968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763622.363046:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763622.363060:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763622.363064:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763622.363072:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3557 00000100:00080000:1.0:1789763622.363078:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763622.363085:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763622.363101:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763622.363127:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565892096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763622.363150:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763622.363159:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763622.363164:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763622.363175:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3557 00000100:00080000:1.0:1789763622.363184:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763622.363193:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763622.363208:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763622.363226:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565891968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763622.363235:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565892224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763622.363268:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763622.363273:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763622.363276:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763622.363283:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3557 00000100:00080000:1.0:1789763622.363289:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763622.363296:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763622.363303:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763622.363319:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565892352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763622.363412:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763622.363489:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763622/real 1789763622] req@ffff93f844cc9180 x1876699565891968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763622.363517:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565891968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763622.363532:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763622.363538:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763622.363543:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565891968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763622.363554:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565892096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763622.363588:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763622.363621:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763622/real 1789763622] req@ffff93f844cc8a80 x1876699565892096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763622.363638:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565892096/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763622.363699:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763622.363704:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763622.363709:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565892096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763622.363747:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565892224:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763622.363775:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763622.363790:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763622/real 1789763622] req@ffff93f844cc9880 x1876699565892224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763622.363805:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565892224/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763622.363816:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763622.363820:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763622.363824:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565892224:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763622.363832:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565892352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763622.363851:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763622.363863:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763622/real 1789763622] req@ffff93f844cc9f80 x1876699565892352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763622.363877:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565892352/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763622.363887:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763622.363890:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763622.363894:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565892352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763622.426911:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763623.450871:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763624.474852:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763625.498832:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763626.522916:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763627.482860:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763627.482878:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763627.482886:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763627.482905:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3562 00000100:00080000:3.0:1789763627.482924:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763627.482942:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763627.483003:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565892480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763627.483070:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763627.483088:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763627.483092:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763627.483101:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3562 00000100:00080000:3.0:1789763627.483110:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763627.483118:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763627.483135:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763627.483162:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565892608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763627.483185:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763627.483191:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763627.483195:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789763627.483205:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565892480:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763627.483208:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3562 00000100:00080000:3.0:1789763627.483223:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763627.483233:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763627.483244:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763627.483273:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565892736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763627.483318:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763627.483325:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763627.483330:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763627.483339:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3562 00000100:00080000:3.0:1789763627.483348:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789763627.483356:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763627.483365:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763627.483380:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763627.483409:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565892864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763627.483436:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763627/real 1789763627] req@ffff93f844d64380 x1876699565892480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763627.483473:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565892480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763627.483494:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763627.483502:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763627.483510:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565892480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763627.483566:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565892608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763627.483617:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763627.483643:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763627/real 1789763627] req@ffff93f844d64000 x1876699565892608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763627.483718:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565892608/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763627.483734:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763627.483741:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763627.483749:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565892608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763627.483761:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565892736:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763627.483805:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763627.483824:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763627/real 1789763627] req@ffff93f844d64a80 x1876699565892736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763627.483843:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565892736/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763627.483880:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763627.483886:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763627.483893:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565892736:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763627.483906:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565892864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763627.483961:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763627.483987:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763627/real 1789763627] req@ffff93f844d65f80 x1876699565892864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763627.484011:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565892864/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763627.484026:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763627.484032:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763627.484040:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565892864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763627.546861:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763628.570831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763629.594838:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763630.618842:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763631.642869:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763632.602812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763632.602826:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763632.602832:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763632.602845:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3567 00000100:00080000:1.0:1789763632.602857:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763632.602866:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763632.602929:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565892992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763632.602967:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763632.602974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763632.602977:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763632.602985:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3567 00000100:00080000:1.0:1789763632.602993:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763632.603002:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763632.603018:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763632.603044:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565893120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763632.603053:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565892992:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763632.603066:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763632.603072:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763632.603075:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763632.603082:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3567 00000100:00080000:1.0:1789763632.603088:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763632.603095:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763632.603104:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763632.603122:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565893248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763632.603141:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763632.603142:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763632.603148:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763632.603151:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763632.603158:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3567 00000100:00080000:1.0:1789763632.603164:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763632.603170:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763632.603178:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763632.603216:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565893376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763632.603216:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763632/real 1789763632] req@ffff93f844cca300 x1876699565892992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763632.603491:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565892992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763632.603507:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763632.603513:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763632.603518:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565892992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763632.603551:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565893120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763632.603588:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763632.603622:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763632/real 1789763632] req@ffff93f844cc8380 x1876699565893120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763632.603639:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565893120/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763632.603690:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763632.603695:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763632.603699:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565893120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763632.603707:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565893248:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763632.603736:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763632.603752:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763632/real 1789763632] req@ffff93f844cc8000 x1876699565893248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763632.603768:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565893248/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763632.603778:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763632.603782:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763632.603786:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565893248:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763632.603792:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565893376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763632.603812:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763632.603823:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763632/real 1789763632] req@ffff93f844ccbb80 x1876699565893376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763632.603838:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565893376/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763632.603848:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763632.603851:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763632.603855:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565893376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763632.666910:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763633.690854:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763634.714934:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763635.738921:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763636.762879:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763637.722833:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763637.722851:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763637.722859:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763637.722879:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3572 00000100:00080000:3.0:1789763637.722900:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763637.722916:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763637.722985:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565893504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763637.723027:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763637.723036:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763637.723040:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763637.723051:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3572 00000100:00080000:3.0:1789763637.723059:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763637.723067:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763637.723084:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763637.723092:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565893504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763637.723108:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565893632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763637.723126:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763637.723132:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763637.723135:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763637.723142:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3572 00000100:00080000:3.0:1789763637.723148:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763637.723155:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763637.723163:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763637.723203:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565893760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763637.723232:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763637.723248:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763637.723254:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763637.723258:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763637.723264:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3572 00000100:00080000:3.0:1789763637.723270:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763637.723277:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763637.723286:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763637.723306:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565893888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763637.723329:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763637/real 1789763637] req@ffff93f844d66300 x1876699565893504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763637.723377:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565893504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763637.723401:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763637.723409:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763637.723418:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565893504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763637.723460:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565893632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763637.723497:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763637.723516:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763637/real 1789763637] req@ffff93f844d65180 x1876699565893632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763637.723532:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565893632/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763637.723552:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763637.723556:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763637.723560:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565893632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763637.723567:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565893760:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763637.723588:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763637.723600:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763637/real 1789763637] req@ffff93f844d65f80 x1876699565893760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763637.723614:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565893760/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763637.723624:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763637.723627:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763637.723631:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565893760:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763637.723637:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565893888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763637.723698:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763637.723712:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763637/real 1789763637] req@ffff93f844d64a80 x1876699565893888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763637.723727:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565893888/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763637.723737:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763637.723741:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763637.723746:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565893888:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763637.786874:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763638.810895:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763639.834823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763640.858911:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763641.882850:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763642.842789:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763642.842803:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763642.842810:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763642.842824:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3577 00000100:00080000:3.0:1789763642.842836:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763642.842846:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763642.842896:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565894016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763642.842935:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763642.842942:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763642.842946:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763642.842956:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3577 00000100:00080000:3.0:1789763642.842964:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763642.842973:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763642.842991:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763642.842993:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565894016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763642.843030:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565894144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763642.843055:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763642.843061:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763642.843065:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763642.843072:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3577 00000100:00080000:3.0:1789763642.843079:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763642.843085:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763642.843093:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789763642.843104:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763642.843114:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565894272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763642.843131:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763642.843136:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763642.843138:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763642.843143:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3577 00000100:00080000:3.0:1789763642.843148:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763642.843153:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763642.843159:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763642.843206:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565894400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763642.843211:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763642/real 1789763642] req@ffff93f844d64000 x1876699565894016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763642.843271:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565894016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763642.843295:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763642.843302:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763642.843310:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565894016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763642.843359:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565894144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763642.843418:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763642.843446:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763642/real 1789763642] req@ffff93f844d64380 x1876699565894144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763642.843473:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565894144/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763642.843493:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763642.843502:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763642.843510:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565894144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763642.843525:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565894272:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763642.843566:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763642.843593:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763642/real 1789763642] req@ffff93f844d65500 x1876699565894272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763642.843614:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565894272/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763642.843626:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763642.843631:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763642.843637:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565894272:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763642.843683:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565894400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763642.843719:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763642.843740:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763642/real 1789763642] req@ffff93f844d66d80 x1876699565894400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763642.843758:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565894400/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763642.843769:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763642.843774:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763642.843781:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565894400:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763642.906833:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763643.930800:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763644.954903:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763645.978883:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763647.002862:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763647.962724:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763647.962735:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763647.962740:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763647.962751:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3582 00000100:00080000:3.0:1789763647.962763:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763647.962772:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763647.962819:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565894528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763647.962856:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763647.962861:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763647.962863:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763647.962868:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3582 00000100:00080000:3.0:1789763647.962872:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763647.962888:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763647.962899:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763647.962915:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565894656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763647.962928:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565894528:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763647.962930:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763647.962941:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763647.962944:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763647.962950:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3582 00000100:00080000:3.0:1789763647.962953:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763647.962956:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763647.962961:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763647.962975:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565894784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763647.962985:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763647.962989:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763647.962992:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763647.962995:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3582 00000100:00080000:3.0:1789763647.963000:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763647.963003:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763647.963007:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763647.963018:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565894912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763647.963054:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565894656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763647.963101:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565894784:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763647.963123:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565894912:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763648.026839:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763649.050848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763650.074880:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1789763651.035189:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1789763651.035279:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1789763651.035307:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789763651.035384:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789763651.035396:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789763651.035412:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789763651.035486:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763647/real 1789763651] req@ffff93f844d67480 x1876699565894656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763651.035540:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565894656/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763651.035563:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763651.035572:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763651.035580:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565894656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763651.035597:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763647/real 1789763651] req@ffff93f844d66a00 x1876699565894784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763651.035622:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565894784/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763651.035640:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763651.035692:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763651.035707:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565894784:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763651.035727:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763647/real 1789763651] req@ffff93f844d64e00 x1876699565894912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763651.035788:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565894912/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763651.035803:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763651.035810:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763651.035816:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565894912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763651.035830:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763647/real 1789763651] req@ffff93f844d65c00 x1876699565894528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763651.035853:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565894528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763651.035867:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763651.035872:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763651.035878:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565894528:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789763651.098836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763652.122903:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763653.082772:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763653.082782:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763653.082786:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763653.082798:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3588 00000100:00080000:1.0:1789763653.082807:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763653.082815:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763653.082856:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565895040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763653.082881:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763653.082887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763653.082888:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763653.082891:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3588 00000100:00080000:1.0:1789763653.082894:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763653.082897:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763653.082907:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763653.082917:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565895168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763653.082930:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763653.082938:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1789763653.082939:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565895040:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763653.082941:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763653.082955:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3588 00000100:00080000:1.0:1789763653.082961:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763653.082968:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763653.082975:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763653.082992:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565895296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763653.083005:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763653.083009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763653.083011:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763653.083017:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3588 00000100:00080000:1.0:1789763653.083041:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789763653.083043:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763653.083057:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763653.083066:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763653.083085:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565895424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763653.083121:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763653/real 1789763653] req@ffff93f844ccb800 x1876699565895040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763653.083162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565895040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763653.083175:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763653.083180:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763653.083184:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565895040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763653.083211:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565895168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763653.083238:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763653.083253:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763653/real 1789763653] req@ffff93f844ccb480 x1876699565895168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763653.083267:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565895168/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763653.083276:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763653.083280:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763653.083283:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565895168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763653.083291:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565895296:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763653.083319:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763653.083332:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763653/real 1789763653] req@ffff93f844ccaa00 x1876699565895296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763653.083344:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565895296/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763653.083353:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763653.083356:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763653.083359:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565895296:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763653.083365:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565895424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763653.083387:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763653.083397:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763653/real 1789763653] req@ffff93f844cc8700 x1876699565895424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763653.083409:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565895424/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763653.083417:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763653.083420:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763653.083423:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565895424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763653.146800:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763654.170910:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763655.194860:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763656.218885:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763657.242932:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763658.202878:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763658.202911:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763658.202917:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763658.202941:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3593 00000100:00080000:0.0:1789763658.202953:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763658.202963:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763658.203033:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565895552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763658.203104:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763658.203113:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763658.203117:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763658.203127:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3593 00000100:00080000:0.0:1789763658.203137:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763658.203149:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763658.203166:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763658.203188:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565895552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763658.203196:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565895680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763658.203229:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763658.203233:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763658.203238:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763658.203247:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3593 00000100:00080000:0.0:1789763658.203253:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763658.203259:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763658.203267:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763658.203284:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565895808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763658.203302:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763658.203310:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763658.203315:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763658.203323:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3593 00000800:00000100:2.0:1789763658.203334:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763658.203335:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763658.203351:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763658.203365:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763658.203398:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565895936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763658.203424:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763658/real 1789763658] req@ffff93f850d40000 x1876699565895552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763658.203458:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565895552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763658.203475:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763658.203482:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763658.203488:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565895552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763658.203527:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565895680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763658.203561:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763658.205463:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763658/real 1789763658] req@ffff93f850d40e00 x1876699565895680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763658.205497:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565895680/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763658.205512:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763658.205517:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763658.205522:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565895680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763658.205535:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565895808:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763658.205605:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763658.205632:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763658/real 1789763658] req@ffff93f850d42680 x1876699565895808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763658.205696:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565895808/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763658.205707:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763658.205711:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763658.205716:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565895808:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763658.205723:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565895936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763658.205752:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763658.205767:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763658/real 1789763658] req@ffff93f850d41c00 x1876699565895936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763658.205782:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565895936/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763658.205803:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763658.205807:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763658.205812:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565895936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763658.266917:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763659.290894:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763660.314831:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763661.338907:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763662.362833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763663.322786:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763663.322860:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763663.322867:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763663.322894:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3598 00000100:00080000:1.0:1789763663.322910:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763663.322921:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763663.322989:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565896064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763663.323067:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763663.323083:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763663.323086:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763663.323093:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3598 00000100:00080000:1.0:1789763663.323100:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763663.323106:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763663.323119:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763663.323153:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565896192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763663.323171:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763663.323177:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763663.323180:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763663.323186:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3598 00000100:00080000:1.0:1789763663.323194:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789763663.323205:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565896064:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763663.323208:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763663.323230:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763663.323260:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565896320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763663.323281:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763663.323286:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763663.323289:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763663.323296:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3598 00000100:00080000:1.0:1789763663.323319:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789763663.323351:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763663.323352:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763663.323368:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763663.323399:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565896448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763663.323441:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763663/real 1789763663] req@ffff93f844ccb100 x1876699565896064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763663.323465:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565896064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763663.323482:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763663.323486:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763663.323490:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565896064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763663.323498:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565896192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763663.323518:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763663.323530:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763663/real 1789763663] req@ffff93f844cca680 x1876699565896192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763663.323546:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565896192/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763663.323553:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763663.323556:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763663.323558:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565896192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763663.323585:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565896320:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763663.323599:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763663.323607:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763663/real 1789763663] req@ffff93f844cc8e00 x1876699565896320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763663.323616:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565896320/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763663.323622:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763663.323625:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763663.323627:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565896320:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763663.323633:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565896448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763663.323680:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763663.323689:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763663/real 1789763663] req@ffff93f844cc9c00 x1876699565896448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763663.323698:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565896448/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763663.323704:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763663.323707:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763663.323710:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565896448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763663.386894:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763664.410898:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763665.434924:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763666.458931:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763667.482828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763668.442783:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763668.442795:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763668.442799:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763668.442811:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3603 00000100:00080000:0.0:1789763668.442824:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763668.442833:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763668.442875:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565896576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763668.442920:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763668.442924:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763668.442927:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763668.442931:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3603 00000100:00080000:0.0:1789763668.442936:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763668.442940:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763668.442950:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763668.442968:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565896704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763668.442968:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565896576:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763668.443013:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763668.443019:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763668.443021:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763668.443046:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3603 00000100:00080000:0.0:1789763668.443057:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763668.443066:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763668.443077:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789763668.443102:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763668.443107:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565896832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763668.443135:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763668.443144:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763668.443149:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763668.443158:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3603 00000100:00100000:3.0:1789763668.443167:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763668/real 1789763668] req@ffff93f850d41180 x1876699565896576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763668.443171:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763668.443189:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763668.443210:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763668.443217:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565896576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763668.443233:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1789763668.443236:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565896960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763668.443243:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763668.443265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565896576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763668.443330:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565896704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763668.443354:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763668.443380:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763668/real 1789763668] req@ffff93f850d43100 x1876699565896704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763668.443394:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565896704/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763668.443403:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763668.443406:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763668.443410:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565896704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763668.443418:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565896832:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763668.443440:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763668.443448:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763668/real 1789763668] req@ffff93f850d43480 x1876699565896832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763668.443461:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565896832/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763668.443467:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763668.443469:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763668.443471:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565896832:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763668.443475:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565896960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763668.443488:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763668.443494:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763668/real 1789763668] req@ffff93f850d41500 x1876699565896960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763668.443502:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565896960/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763668.443506:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763668.443508:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763668.443510:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565896960:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763668.506879:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763669.530960:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763670.554830:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763671.578921:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763672.602905:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763673.562839:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763673.562855:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763673.562863:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763673.562884:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3608 00000100:00080000:3.0:1789763673.562906:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763673.562922:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763673.563038:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565897088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763673.563104:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763673.563110:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763673.563112:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763673.563119:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3608 00000100:00080000:3.0:1789763673.563128:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763673.563137:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763673.563154:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763673.563179:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565897216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763673.563200:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565897088:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763673.563201:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763673.563223:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763673.563227:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763673.563234:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3608 00000100:00080000:3.0:1789763673.563239:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763673.563245:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763673.563254:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763673.563273:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565897344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763673.563294:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763673.563300:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763673.563303:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763673.563309:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3608 00000100:00080000:3.0:1789763673.563315:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789763673.563328:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763673.563338:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763673.563347:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763673.563362:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565897472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763673.563453:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763673/real 1789763673] req@ffff93f844d67100 x1876699565897088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763673.563485:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565897088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763673.563501:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763673.563507:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763673.563513:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565897088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763673.563549:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565897216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763673.563588:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763673.563606:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763673/real 1789763673] req@ffff93f844d67800 x1876699565897216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763673.563624:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565897216/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763673.563635:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763673.563640:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763673.563678:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565897216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763673.563688:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565897344:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763673.563716:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763673.563731:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763673/real 1789763673] req@ffff93f844d64700 x1876699565897344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763673.563746:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565897344/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763673.563757:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763673.563761:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763673.563765:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565897344:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763673.563772:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565897472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763673.563794:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763673.563806:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763673/real 1789763673] req@ffff93f844d67b80 x1876699565897472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763673.563821:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565897472/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763673.563839:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763673.563843:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763673.563847:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565897472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763673.626864:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763674.650881:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763675.674881:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763676.698870:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763677.722862:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763678.682965:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763678.682977:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763678.682984:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763678.683001:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3613 00000100:00080000:1.0:1789763678.683019:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763678.683033:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763678.683094:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccad80 x1876699565897600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763678.683141:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763678.683151:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763678.683154:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763678.683160:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3613 00000100:00080000:1.0:1789763678.683166:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763678.683176:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763678.683180:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565897600:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1789763678.683192:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763678.683224:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9500 x1876699565897728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763678.683246:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763678.683254:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763678.683260:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763678.683273:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3613 00000100:00080000:1.0:1789763678.683282:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763678.683291:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763678.683303:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789763678.683322:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763678.683335:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565897856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763678.683360:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763678.683366:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763678.683370:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763678.683377:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3613 00000100:00080000:1.0:1789763678.683384:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763678.683390:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763678.683400:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763678.683421:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565897984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763678.683431:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763678/real 1789763678] req@ffff93f844ccad80 x1876699565897600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763678.683472:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccad80 x1876699565897600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763678.683491:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763678.683498:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763678.683507:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565897600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763678.683550:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565897728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763678.683589:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763678.683608:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763678/real 1789763678] req@ffff93f844cc9500 x1876699565897728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763678.683625:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9500 x1876699565897728/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763678.683637:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763678.683693:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763678.683699:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565897728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763678.683711:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565897856:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763678.683749:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763678.683774:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763678/real 1789763678] req@ffff93f844cc9c00 x1876699565897856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763678.683793:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565897856/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763678.683808:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763678.683814:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763678.683818:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565897856:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763678.683829:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565897984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763678.683856:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763678.683877:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763678/real 1789763678] req@ffff93f844cc8e00 x1876699565897984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763678.683897:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565897984/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763678.683941:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763678.683952:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763678.683963:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565897984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763678.746886:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763679.770861:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763680.794821:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763681.818863:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763682.842921:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763683.802723:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763683.802730:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763683.802734:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763683.802742:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3618 00000100:00080000:1.0:1789763683.802749:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763683.802755:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763683.802786:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565898112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763683.802807:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763683.802810:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763683.802811:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763683.802814:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3618 00000100:00080000:1.0:1789763683.802816:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763683.802818:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763683.802825:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763683.802833:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565898240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763683.802841:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763683.802844:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763683.802845:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763683.802848:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3618 00000100:00080000:1.0:1789763683.802851:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763683.802853:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763683.802857:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763683.802864:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565898368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763683.802883:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565898112:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763683.802887:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763683.802891:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763683.802892:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763683.802895:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3618 00000100:00080000:1.0:1789763683.802898:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763683.802901:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763683.802904:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763683.802915:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565898496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763683.802943:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763683.802984:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763683/real 1789763683] req@ffff93f844cca680 x1876699565898112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763683.803000:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565898112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763683.803010:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763683.803013:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763683.803017:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565898112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763683.803022:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565898240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763683.803038:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763683.803046:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763683/real 1789763683] req@ffff93f844ccb100 x1876699565898240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763683.803054:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565898240/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763683.803059:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763683.803061:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763683.803063:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565898240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763683.803082:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565898368:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763683.803093:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763683.803100:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763683/real 1789763683] req@ffff93f844cc8700 x1876699565898368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763683.803107:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565898368/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763683.803116:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763683.803118:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763683.803121:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565898368:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763683.803124:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565898496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763683.803134:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763683.803140:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763683/real 1789763683] req@ffff93f844ccaa00 x1876699565898496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763683.803148:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565898496/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763683.803153:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763683.803155:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763683.803156:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565898496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763683.866777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763684.890937:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763685.914876:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763686.938928:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763687.962937:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763688.922799:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763688.922812:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763688.922818:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763688.922831:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3623 00000100:00080000:2.0:1789763688.922869:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763688.922879:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763688.922931:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565898624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763688.922976:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763688.922982:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763688.922985:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763688.922990:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3623 00000100:00080000:2.0:1789763688.922995:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763688.923002:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763688.923014:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763688.923038:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565898624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763688.923039:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565898752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763688.923068:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763688.923072:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763688.923074:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763688.923079:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3623 00000100:00080000:2.0:1789763688.923083:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763688.923088:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763688.923095:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763688.923111:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565898880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763688.923124:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763688.923131:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763688.923134:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763688.923150:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3623 00000800:00000100:3.0:1789763688.923151:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763688.923166:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763688.923178:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763688.923192:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763688.923218:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565899008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763688.923248:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763688/real 1789763688] req@ffff93f8517c0e00 x1876699565898624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763688.923291:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565898624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763688.923314:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763688.923320:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763688.923326:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565898624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763688.923367:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565898752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763688.923416:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763688.923458:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763688/real 1789763688] req@ffff93f8517c3800 x1876699565898752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763688.923476:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565898752/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763688.923489:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763688.923495:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763688.923503:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565898752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763688.923515:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565898880:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763688.923557:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763688.923580:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763688/real 1789763688] req@ffff93f8517c3480 x1876699565898880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763688.923605:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565898880/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763688.923625:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763688.923630:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763688.923634:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565898880:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763688.923642:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565899008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763688.923707:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763688.923723:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763688/real 1789763688] req@ffff93f8517c1880 x1876699565899008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763688.923738:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565899008/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763688.923749:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763688.923753:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763688.923756:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565899008:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763688.986857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763690.010916:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763691.034869:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763692.058879:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763693.082965:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763694.043268:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763694.043293:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763694.043301:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763694.043319:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3629 00000100:00080000:1.0:1789763694.043336:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763694.043350:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763694.043410:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565899136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763694.043539:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565899136:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1789763694.043700:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763694.043833:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763694.043847:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763694.043852:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763694.043863:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3629 00000100:00080000:1.0:1789763694.043874:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763694.043885:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763694.043901:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763694.043932:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565899264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763694.044044:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763694/real 1789763694] req@ffff93f844ccb480 x1876699565899136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763694.044077:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565899136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763694.044092:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763694.044098:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763694.044106:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565899136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763694.044146:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565899264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763694.044198:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763694.044226:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763694/real 1789763694] req@ffff93f844ccb800 x1876699565899264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763694.044253:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565899264/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763694.044271:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763694.044279:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763694.044286:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565899264:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1789763694.044376:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763694.044385:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763694.044390:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763694.044399:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3629 00000100:00080000:1.0:1789763694.044408:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763694.044417:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763694.044428:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763694.044455:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565899392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763694.044478:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763694.044484:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763694.044487:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763694.044494:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3629 00000100:00080000:1.0:1789763694.044501:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763694.044508:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763694.044516:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763694.044531:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565899520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763694.044625:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565899392:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763694.044692:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763694.044712:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763694/real 1789763694] req@ffff93f844ccbb80 x1876699565899392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763694.044729:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565899392/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763694.044741:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763694.044745:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763694.044750:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565899392:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763694.044758:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565899520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763694.044783:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763694.044824:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763694/real 1789763694] req@ffff93f844cc8000 x1876699565899520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763694.044845:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565899520/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763694.044857:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763694.044861:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763694.044865:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565899520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763694.106870:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763695.130894:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763696.154912:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763697.178944:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763698.202980:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763699.162870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763699.162889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763699.162897:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763699.162913:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3634 00000100:00080000:1.0:1789763699.162931:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763699.162942:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763699.162993:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565899648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763699.163033:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763699.163038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763699.163041:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763699.163046:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3634 00000100:00080000:1.0:1789763699.163051:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763699.163056:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763699.163069:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763699.163086:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565899776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763699.163105:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1789763699.163121:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565899648:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763699.163124:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763699.163133:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763699.163155:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3634 00000100:00080000:1.0:1789763699.163162:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763699.163168:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763699.163175:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763699.163195:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565899904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763699.163209:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763699.163214:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763699.163219:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763699.163224:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3634 00000100:00080000:1.0:1789763699.163229:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763699.163233:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763699.163242:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763699.163258:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565900032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763699.163276:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763699.163371:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763699/real 1789763699] req@ffff93f844cc8380 x1876699565899648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763699.163403:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565899648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763699.163422:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763699.163429:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763699.163434:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565899648:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763699.163475:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565899776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763699.163505:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763699.163521:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763699/real 1789763699] req@ffff93f844cca300 x1876699565899776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763699.163537:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565899776/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763699.163548:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763699.163552:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763699.163556:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565899776:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763699.163564:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565899904:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763699.163586:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763699.163598:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763699/real 1789763699] req@ffff93f844cc9f80 x1876699565899904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763699.163612:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565899904/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763699.163622:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763699.163626:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763699.163629:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565899904:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763699.163636:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565900032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763699.163696:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763699.163712:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763699/real 1789763699] req@ffff93f844cc9880 x1876699565900032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763699.163727:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565900032/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763699.163738:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763699.163742:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763699.163745:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565900032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763699.226912:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763700.250911:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763701.274880:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763702.298918:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763703.322901:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763704.282839:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763704.282852:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763704.282857:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763704.282868:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3639 00000100:00080000:2.0:1789763704.282879:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763704.282886:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763704.282929:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565900160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763704.282957:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763704.282960:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763704.282962:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763704.282965:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3639 00000100:00080000:2.0:1789763704.282968:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763704.282971:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763704.282981:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763704.282990:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565900160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763704.282992:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565900288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763704.283005:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763704.283008:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763704.283009:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763704.283012:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3639 00000100:00080000:2.0:1789763704.283015:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763704.283034:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763704.283040:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763704.283051:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565900416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763704.283061:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763704.283065:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763704.283067:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763704.283071:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3639 00000100:00080000:2.0:1789763704.283085:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763704.283089:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763704.283111:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789763704.283122:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763704.283130:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565900544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763704.283196:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763704/real 1789763704] req@ffff93f8517c3100 x1876699565900160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763704.283223:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565900160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763704.283243:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763704.283251:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763704.283272:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565900160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763704.283302:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565900288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763704.283325:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763704.283335:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763704/real 1789763704] req@ffff93f8517c1180 x1876699565900288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763704.283344:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565900288/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763704.283350:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763704.283353:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763704.283355:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565900288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763704.283359:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565900416:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763704.283372:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763704.283378:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763704/real 1789763704] req@ffff93f8517c2a00 x1876699565900416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763704.283386:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565900416/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763704.283391:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763704.283392:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763704.283394:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565900416:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763704.283398:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565900544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763704.283408:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763704.283414:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763704/real 1789763704] req@ffff93f8517c2d80 x1876699565900544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763704.283421:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565900544/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763704.283430:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763704.283432:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763704.283434:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565900544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763704.346897:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763705.370836:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763706.394851:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763707.418884:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763708.442899:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763709.402801:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763709.402817:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763709.402824:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763709.402842:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3644 00000100:00080000:2.0:1789763709.402859:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763709.402872:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763709.402935:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565900672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763709.402978:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763709.402987:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763709.402990:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763709.402995:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3644 00000100:00080000:2.0:1789763709.403000:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763709.403005:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763709.403020:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763709.403037:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565900800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763709.403053:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565900672:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763709.403057:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763709.403069:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763709.403072:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763709.403079:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3644 00000100:00080000:2.0:1789763709.403083:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763709.403088:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763709.403094:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763709.403109:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565900928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763709.403124:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763709.403130:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763709.403133:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763709.403137:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3644 00000100:00080000:2.0:1789763709.403154:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763709.403159:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763709.403166:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789763709.403176:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763709.403187:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565901056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763709.403262:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763709/real 1789763709] req@ffff93f8517c0a80 x1876699565900672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763709.403291:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565900672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763709.403306:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763709.403312:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763709.403318:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565900672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763709.403360:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565900800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763709.403399:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763709.403425:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763709/real 1789763709] req@ffff93f8517c0380 x1876699565900800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763709.403451:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565900800/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763709.403470:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763709.403476:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763709.403483:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565900800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763709.403497:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565900928:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763709.403542:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763709.403568:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763709/real 1789763709] req@ffff93f8517c2680 x1876699565900928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763709.403594:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565900928/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763709.403611:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763709.403618:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763709.403624:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565900928:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763709.403637:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565901056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763709.403753:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763709.403779:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763709/real 1789763709] req@ffff93f8517c1500 x1876699565901056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763709.403803:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565901056/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763709.403815:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763709.403819:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763709.403824:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565901056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763709.466930:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763710.491024:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763711.514908:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763712.538955:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763713.562954:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763714.522879:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763714.522895:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763714.522903:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763714.522933:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3649 00000100:00080000:2.0:1789763714.522952:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763714.522966:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763714.523043:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565901184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763714.523088:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763714.523094:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763714.523098:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763714.523107:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3649 00000100:00080000:2.0:1789763714.523113:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763714.523119:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763714.523132:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763714.523135:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565901184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763714.523161:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565901312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763714.523190:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763714.523198:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763714.523204:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763714.523216:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3649 00000100:00080000:2.0:1789763714.523231:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763714.523242:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763714.523258:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763714.523296:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565901312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763714.523297:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565901440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763714.523347:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763714.523352:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763714.523356:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763714.523364:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3649 00000100:00080000:2.0:1789763714.523370:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763714.523377:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763714.523387:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763714.523408:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565901568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763714.524876:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565901440:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763714.524929:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565901568:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763714.586873:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763715.610904:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763716.634911:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1789763717.595078:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1789763717.595162:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1789763717.595216:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789763717.595287:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789763717.595308:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1789763717.595328:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1789763717.595380:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763714/real 1789763717] req@ffff93f8517c3b80 x1876699565901440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763717.595437:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565901440/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763717.595465:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763717.595476:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763717.595484:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565901440:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763717.595500:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763714/real 1789763717] req@ffff93f8517c0000 x1876699565901568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763717.595538:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565901568/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763717.595554:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763717.595560:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763717.595568:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565901568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763717.595582:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763714/real 1789763717] req@ffff93f8517c1f80 x1876699565901312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763717.595606:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565901312/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763717.595683:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763717.595692:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763717.595701:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565901312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763717.595719:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763714/real 1789763717] req@ffff93f8517c2300 x1876699565901184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763717.595747:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565901184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763717.595764:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763717.595770:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763717.595777:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565901184:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789763717.658853:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763718.682831:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763719.642796:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763719.642808:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763719.642815:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763719.642829:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3654 00000100:00080000:2.0:1789763719.642843:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763719.642853:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763719.642900:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565901696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763719.642951:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763719.642961:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763719.642964:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763719.642971:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3654 00000100:00080000:2.0:1789763719.642977:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763719.642982:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763719.642995:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763719.643014:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565901824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763719.643030:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763719.643039:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:3.0:1789763719.643041:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565901696:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763719.643043:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763719.643058:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3654 00000100:00080000:2.0:1789763719.643063:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763719.643067:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763719.643074:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763719.643094:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565901952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763719.643115:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763719.643122:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763719.643126:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763719.643138:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3654 00000800:00000100:3.0:1789763719.643140:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763719.643151:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763719.643175:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763719.643189:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763719.643214:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565902080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763719.643242:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763719/real 1789763719] req@ffff93f8517c2300 x1876699565901696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763719.643297:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565901696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763719.643320:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763719.643326:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763719.643331:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565901696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763719.643369:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565901824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763719.643415:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763719.643442:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763719/real 1789763719] req@ffff93f8517c1f80 x1876699565901824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763719.643467:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565901824/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763719.643484:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763719.643491:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763719.643499:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565901824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763719.643512:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565901952:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763719.643556:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763719.643582:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763719/real 1789763719] req@ffff93f8517c0000 x1876699565901952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763719.643685:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565901952/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763719.643703:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763719.643708:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763719.643713:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565901952:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763719.643722:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565902080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763719.643769:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763719.643792:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763719/real 1789763719] req@ffff93f8517c3b80 x1876699565902080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763719.643816:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565902080/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763719.643833:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763719.643840:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763719.643846:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565902080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763719.706833:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763720.730838:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763721.754860:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763722.778790:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763723.802930:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763724.762766:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763724.762779:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763724.762784:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763724.762796:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3659 00000100:00080000:2.0:1789763724.762809:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763724.762818:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763724.762864:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565902208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763724.762903:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763724.762908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763724.762910:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763724.762916:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3659 00000100:00080000:2.0:1789763724.762920:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763724.762924:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763724.762935:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763724.762951:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565902336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763724.762963:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763724.762968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763724.762970:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763724.762974:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3659 00000100:00080000:2.0:1789763724.762979:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763724.762984:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763724.762990:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763724.763003:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565902464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763724.763016:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763724.763020:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763724.763023:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763724.763029:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3659 00000100:00080000:2.0:1789763724.763033:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1789763724.763042:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565902208:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763724.763045:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763724.763059:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763724.763076:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565902592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763724.763155:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763724.763236:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763724/real 1789763724] req@ffff93f8517c1500 x1876699565902208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763724.763264:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565902208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763724.763280:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763724.763286:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763724.763293:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565902208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763724.763307:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565902336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763724.763341:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763724.763358:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763724/real 1789763724] req@ffff93f8517c2680 x1876699565902336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763724.763373:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565902336/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763724.763384:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763724.763388:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763724.763393:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565902336:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763724.763399:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565902464:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763724.763419:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763724.763429:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763724/real 1789763724] req@ffff93f8517c0380 x1876699565902464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763724.763444:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565902464/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763724.763462:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763724.763466:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763724.763470:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565902464:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763724.763501:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565902592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763724.763521:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763724.763533:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763724/real 1789763724] req@ffff93f8517c0a80 x1876699565902592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763724.763547:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565902592/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763724.763558:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763724.763561:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763724.763565:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565902592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763724.826833:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763725.850917:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763726.874902:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763727.898841:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763728.922847:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763729.882848:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763729.882865:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763729.882872:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763729.882892:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3664 00000100:00080000:2.0:1789763729.882911:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763729.882925:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763729.882978:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565902720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763729.883021:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763729.883030:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763729.883032:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763729.883037:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3664 00000100:00080000:2.0:1789763729.883043:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763729.883049:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763729.883062:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763729.883079:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565902848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763729.883095:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763729.883099:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763729.883102:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763729.883115:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3664 00000100:00080000:2.0:1789763729.883125:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1789763729.883125:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565902720:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763729.883139:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763729.883157:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763729.883172:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565902976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763729.883188:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763729.883193:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763729.883196:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763729.883202:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3664 00000100:00080000:2.0:1789763729.883209:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763729.883213:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763729.883223:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763729.883237:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565903104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763729.883284:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763729.883365:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763729/real 1789763729] req@ffff93f8517c2d80 x1876699565902720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763729.883395:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565902720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763729.883415:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763729.883422:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763729.883428:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565902720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763729.883467:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565902848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763729.883497:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763729.883514:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763729/real 1789763729] req@ffff93f8517c2a00 x1876699565902848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763729.883530:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565902848/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763729.883576:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763729.883581:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763729.883586:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565902848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763729.883596:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565902976:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763729.883628:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763729.883642:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763729/real 1789763729] req@ffff93f8517c1180 x1876699565902976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763729.883691:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565902976/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763729.883715:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763729.883724:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763729.883729:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565902976:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763729.883738:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565903104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763729.883765:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763729.883779:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763729/real 1789763729] req@ffff93f8517c3100 x1876699565903104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763729.883794:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565903104/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763729.883804:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763729.883808:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763729.883812:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565903104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763729.946905:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763730.970988:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763731.994893:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763733.018979:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763734.042956:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763735.002835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763735.002856:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763735.002865:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763735.002882:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3669 00000100:00080000:2.0:1789763735.002898:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763735.002910:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763735.002986:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565903232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763735.003028:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763735.003033:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763735.003035:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763735.003040:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3669 00000100:00080000:2.0:1789763735.003045:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763735.003050:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763735.003064:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763735.003082:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565903360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763735.003097:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565903232:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763735.003106:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763735.003119:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763735.003122:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763735.003128:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3669 00000100:00080000:2.0:1789763735.003133:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763735.003137:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763735.003145:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763735.003162:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565903488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763735.003176:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763735.003183:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763735.003188:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763735.003195:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3669 00000100:00080000:2.0:1789763735.003204:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763735.003213:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763735.003223:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789763735.003232:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763735.003258:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565903616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763735.003315:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763735/real 1789763735] req@ffff93f8517c1880 x1876699565903232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763735.003340:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565903232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763735.003355:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763735.003360:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763735.003364:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565903232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763735.003397:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565903360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763735.003429:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763735.003443:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763735/real 1789763735] req@ffff93f8517c3480 x1876699565903360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763735.003457:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565903360/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763735.003467:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763735.003471:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763735.003475:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565903360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763735.003482:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565903488:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763735.003530:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763735.003544:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763735/real 1789763735] req@ffff93f8517c3800 x1876699565903488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763735.003558:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565903488/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763735.003568:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763735.003571:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763735.003575:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565903488:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763735.003582:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565903616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763735.003604:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763735.003614:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763735/real 1789763735] req@ffff93f8517c0e00 x1876699565903616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763735.003636:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565903616/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763735.003672:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763735.003676:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763735.003680:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565903616:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763735.066844:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763736.090873:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763737.114930:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763738.138823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763739.162820:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763740.122826:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763740.122843:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763740.122851:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763740.122868:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3675 00000100:00080000:2.0:1789763740.122884:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763740.122898:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763740.122963:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565903744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763740.123001:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763740.123010:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763740.123012:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763740.123017:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3675 00000100:00080000:2.0:1789763740.123024:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763740.123030:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763740.123044:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763740.123070:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565903744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763740.123071:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565903872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763740.123099:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763740.123103:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763740.123105:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763740.123111:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3675 00000100:00080000:2.0:1789763740.123116:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763740.123120:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763740.123127:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763740.123144:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565904000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763740.123157:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763740.123162:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763740.123165:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1789763740.123174:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763740.123180:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3675 00000100:00080000:2.0:1789763740.123187:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763740.123193:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763740.123201:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763740.123230:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565904128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763740.123239:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763740/real 1789763740] req@ffff93f8517c0e00 x1876699565903744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763740.123272:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565903744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763740.123288:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763740.123294:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763740.123300:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565903744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763740.123329:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565903872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763740.123356:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763740.123371:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763740/real 1789763740] req@ffff93f8517c3800 x1876699565903872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763740.123397:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565903872/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763740.123414:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763740.123420:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763740.123427:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565903872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763740.123439:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565904000:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763740.123508:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763740.123533:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763740/real 1789763740] req@ffff93f8517c3480 x1876699565904000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763740.123557:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565904000/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763740.123568:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763740.123573:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763740.123577:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565904000:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763740.123585:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565904128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763740.123614:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763740.123628:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763740/real 1789763740] req@ffff93f8517c1880 x1876699565904128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763740.123682:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565904128/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763740.123703:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763740.123708:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763740.123712:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565904128:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763740.186830:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763741.210924:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763742.234869:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763743.258860:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763744.282902:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763745.242871:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763745.242883:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763745.242889:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763745.242902:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3680 00000100:00080000:1.0:1789763745.242919:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763745.242931:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763745.242996:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8a80 x1876699565904256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763745.243057:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763745.243062:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763745.243065:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763745.243071:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3680 00000100:00080000:1.0:1789763745.243076:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763745.243083:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763745.243106:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763745.243128:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9180 x1876699565904384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763745.243146:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763745.243156:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763745.243160:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1789763745.243160:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565904256:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763745.243170:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3680 00000100:00080000:1.0:1789763745.243187:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763745.243195:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763745.243200:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763745.243226:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9880 x1876699565904512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763745.243238:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763745.243242:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763745.243244:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763745.243249:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3680 00000100:00080000:1.0:1789763745.243258:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763745.243265:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763745.243271:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763745.243283:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9f80 x1876699565904640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763745.243297:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763745.243381:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763745/real 1789763745] req@ffff93f844cc8a80 x1876699565904256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763745.243407:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8a80 x1876699565904256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763745.243442:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763745.243466:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763745.243471:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565904256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763745.243509:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565904384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763745.243559:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763745.243575:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763745/real 1789763745] req@ffff93f844cc9180 x1876699565904384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763745.243589:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9180 x1876699565904384/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763745.243606:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763745.243610:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763745.243614:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565904384:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763745.243620:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565904512:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763745.243642:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763745.243652:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763745/real 1789763745] req@ffff93f844cc9880 x1876699565904512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763745.243664:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9880 x1876699565904512/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763745.243703:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763745.243708:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763745.243711:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565904512:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763745.243719:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565904640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763745.243742:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763745.243772:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763745/real 1789763745] req@ffff93f844cc9f80 x1876699565904640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763745.243784:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9f80 x1876699565904640/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763745.243793:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763745.243796:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763745.243800:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565904640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763745.306838:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763746.330814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763747.354911:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763748.378946:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763749.402910:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763750.362816:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763750.362829:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763750.362833:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763750.362856:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3685 00000100:00080000:3.0:1789763750.362869:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763750.362880:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763750.362925:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565904768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763750.362959:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763750.362964:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763750.362966:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763750.362971:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3685 00000100:00080000:3.0:1789763750.362975:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763750.362979:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763750.362990:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763750.363014:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565904896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763750.363030:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763750.363038:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763750.363042:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763750.363054:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3685 00000100:00080000:3.0:1789763750.363064:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763750.363075:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763750.363086:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565904768:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789763750.363094:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763750.363172:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565905024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763750.363204:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763750.363210:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763750.363215:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789763750.363248:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763750.363331:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763750/real 1789763750] req@ffff93f844d66680 x1876699565904768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763750.363350:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3685 00000100:00100000:0.0:1789763750.363375:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565904768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763750.363436:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763750.363446:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763750.363455:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565904768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763750.363473:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565904896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763750.363542:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763750.363571:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763750/real 1789763750] req@ffff93f844d65880 x1876699565904896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763750.363591:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565904896/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763750.363605:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763750.363610:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763750.363615:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565904896:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1789763750.363626:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763750.363706:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763750.363742:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763750.363753:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565905024:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763750.363809:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763750.363852:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565905152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763750.363856:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763750/real 1789763750] req@ffff93f844d67b80 x1876699565905024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763750.364003:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565905024/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763750.364016:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763750.364021:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763750.364026:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565905024:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763750.364047:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565905152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763750.364083:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763750.364096:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763750/real 1789763750] req@ffff93f844d64700 x1876699565905152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763750.364116:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565905152/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763750.364142:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763750.364148:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763750.364156:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565905152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763750.426903:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763751.450882:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763752.474955:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763753.498884:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763754.522884:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763755.482821:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763755.482842:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763755.482850:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763755.482870:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3690 00000100:00080000:1.0:1789763755.482888:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763755.482900:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763755.482976:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca300 x1876699565905280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763755.483027:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763755.483033:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763755.483036:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763755.483042:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3690 00000100:00080000:1.0:1789763755.483048:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763755.483054:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763755.483070:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763755.483101:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8380 x1876699565905408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763755.483125:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789763755.483136:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565905280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763755.483137:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763755.483149:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763755.483161:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3690 00000100:00080000:1.0:1789763755.483169:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763755.483176:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763755.483184:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763755.483207:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8000 x1876699565905536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763755.483225:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763755.483231:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763755.483234:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1789763755.483240:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1789763755.483246:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3690 00000100:00080000:1.0:1789763755.483254:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763755.483260:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763755.483268:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763755.483282:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccbb80 x1876699565905664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763755.483327:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763755/real 1789763755] req@ffff93f844cca300 x1876699565905280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763755.483383:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca300 x1876699565905280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763755.483404:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763755.483409:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763755.483413:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565905280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763755.483442:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565905408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763755.483473:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763755.483488:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763755/real 1789763755] req@ffff93f844cc8380 x1876699565905408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763755.483500:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8380 x1876699565905408/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763755.483509:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763755.483513:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763755.483516:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565905408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763755.483522:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565905536:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763755.483540:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763755.483551:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763755/real 1789763755] req@ffff93f844cc8000 x1876699565905536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763755.483562:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8000 x1876699565905536/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763755.483571:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763755.483574:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763755.483577:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565905536:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763755.483583:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565905664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763755.483599:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763755.483609:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763755/real 1789763755] req@ffff93f844ccbb80 x1876699565905664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763755.483621:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccbb80 x1876699565905664/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763755.483629:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763755.483632:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763755.483635:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565905664:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763755.546843:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763756.570909:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763757.594885:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763758.618869:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763759.642865:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763760.603229:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763760.603251:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763760.603259:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763760.603279:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3695 00000100:00080000:3.0:1789763760.603297:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763760.603329:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763760.603396:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565905792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763760.603430:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763760.603436:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763760.603439:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763760.603445:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3695 00000100:00080000:3.0:1789763760.603451:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763760.603456:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763760.603472:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763760.603490:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565905920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763760.603507:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763760.603513:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763760.603515:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763760.603522:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3695 00000100:00080000:3.0:1789763760.603527:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763760.603533:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763760.603540:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763760.603561:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565906048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763760.603589:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763760.603595:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763760.603598:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763760.603605:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3695 00000100:00080000:3.0:1789763760.603610:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763760.603617:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763760.603625:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763760.603693:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565906176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763760.603964:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565905792:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1789763760.604094:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763760.604159:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763760/real 1789763760] req@ffff93f844d67800 x1876699565905792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763760.604183:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565905792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763760.604197:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763760.604202:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763760.604207:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565905792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763760.604216:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565905920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763760.604245:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763760.604259:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763760/real 1789763760] req@ffff93f844d67100 x1876699565905920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763760.604273:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565905920/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763760.604283:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763760.604286:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763760.604290:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565905920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763760.604296:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565906048:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763760.604347:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763760.604361:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763760/real 1789763760] req@ffff93f844d65c00 x1876699565906048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763760.604374:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565906048/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763760.604383:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763760.604387:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763760.604390:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565906048:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763760.604397:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565906176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763760.604416:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763760.604430:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763760/real 1789763760] req@ffff93f844d64e00 x1876699565906176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763760.604451:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565906176/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763760.604476:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763760.604482:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763760.604488:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565906176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763760.666836:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763761.690821:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763762.714816:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763763.738831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763764.762851:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763765.722797:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763765.722809:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763765.722815:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763765.722828:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3700 00000100:00080000:2.0:1789763765.722840:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763765.722850:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763765.722899:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565906304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763765.722939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763765.722944:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763765.722946:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763765.722952:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3700 00000100:00080000:2.0:1789763765.722956:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763765.722960:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763765.722970:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763765.722987:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565906432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763765.723001:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763765.723006:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763765.723009:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763765.723014:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3700 00000100:00100000:0.0:1789763765.723024:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565906304:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763765.723025:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763765.723039:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763765.723047:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763765.723061:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565906560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763765.723075:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763765.723080:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763765.723083:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763765.723087:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3700 00000100:00080000:2.0:1789763765.723094:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763765.723097:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763765.723103:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763765.723118:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565906688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763765.723129:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763765.723205:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763765/real 1789763765] req@ffff93f8517c3100 x1876699565906304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763765.723233:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565906304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763765.723248:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763765.723255:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763765.723261:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565906304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763765.723301:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565906432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763765.723337:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763765.723354:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763765/real 1789763765] req@ffff93f8517c1180 x1876699565906432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763765.723370:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565906432/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763765.723381:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763765.723385:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763765.723390:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565906432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763765.723420:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565906560:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763765.723443:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763765.723455:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763765/real 1789763765] req@ffff93f8517c2a00 x1876699565906560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763765.723470:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565906560/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763765.723482:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763765.723487:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763765.723493:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565906560:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763765.723504:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565906688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763765.723543:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763765.723566:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763765/real 1789763765] req@ffff93f8517c2d80 x1876699565906688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763765.723591:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565906688/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763765.723606:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763765.723611:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763765.723615:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565906688:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763765.786820:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763766.810830:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763767.834826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763768.858874:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763769.882790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763770.842730:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763770.842741:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763770.842746:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763770.842755:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3705 00000100:00080000:2.0:1789763770.842764:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763770.842769:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763770.842801:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565906816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763770.842824:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763770.842829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763770.842831:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763770.842834:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3705 00000100:00080000:2.0:1789763770.842836:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763770.842839:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763770.842846:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763770.842855:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565906944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763770.842861:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763770.842864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763770.842865:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763770.842867:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3705 00000100:00080000:2.0:1789763770.842869:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763770.842871:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763770.842874:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763770.842881:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565907072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763770.842888:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763770.842890:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763770.842900:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789763770.842902:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565906816:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763770.842906:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3705 00000100:00080000:2.0:1789763770.842910:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763770.842912:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763770.842916:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763770.842927:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565907200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763770.842962:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763770.842996:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763770/real 1789763770] req@ffff93f8517c0a80 x1876699565906816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763770.843010:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565906816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763770.843018:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763770.843021:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763770.843024:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565906816:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763770.843028:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565906944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763770.843042:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763770.843050:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763770/real 1789763770] req@ffff93f8517c0380 x1876699565906944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763770.843058:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565906944/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763770.843063:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763770.843065:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763770.843067:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565906944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763770.843071:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565907072:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763770.843081:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763770.843087:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763770/real 1789763770] req@ffff93f8517c2680 x1876699565907072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763770.843094:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565907072/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763770.843104:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763770.843106:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763770.843108:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565907072:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763770.843125:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565907200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763770.843136:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763770.843142:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763770/real 1789763770] req@ffff93f8517c1500 x1876699565907200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763770.843149:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565907200/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763770.843154:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763770.843156:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763770.843157:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565907200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763770.906872:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763771.930761:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763772.954844:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763773.978888:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763775.002818:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763775.962793:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763775.962805:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763775.962811:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763775.962825:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3710 00000100:00080000:2.0:1789763775.962838:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763775.962848:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763775.962893:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565907328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763775.962919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763775.962922:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763775.962924:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763775.962928:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3710 00000100:00080000:2.0:1789763775.962933:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763775.962938:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763775.962948:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763775.962956:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565907328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763775.962963:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565907456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763775.962974:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763775.962977:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763775.962979:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763775.962982:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3710 00000100:00080000:2.0:1789763775.962986:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763775.962988:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763775.962993:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763775.963020:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565907584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763775.963031:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763775.963034:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763775.963035:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763775.963038:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3710 00000100:00080000:2.0:1789763775.963042:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789763775.963048:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763775.963053:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763775.963059:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763775.963071:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565907712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763775.963127:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763775/real 1789763775] req@ffff93f8517c3b80 x1876699565907328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763775.963165:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565907328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763775.963176:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763775.963180:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763775.963184:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565907328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763775.963227:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565907456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763775.963282:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763775.963296:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763775/real 1789763775] req@ffff93f8517c0000 x1876699565907456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763775.963308:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565907456/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763775.963316:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763775.963319:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763775.963322:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565907456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763775.963327:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565907584:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763775.963344:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763775.963352:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763775/real 1789763775] req@ffff93f8517c1f80 x1876699565907584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763775.963362:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565907584/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763775.963369:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763775.963371:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763775.963374:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565907584:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763775.963378:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565907712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763775.963392:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763775.963400:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763775/real 1789763775] req@ffff93f8517c2300 x1876699565907712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763775.963410:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565907712/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763775.963416:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763775.963419:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763775.963421:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565907712:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1789763776.026763:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763777.050826:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763778.074782:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789763779.098784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763780.122761:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763781.082758:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763781.082772:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763781.082778:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763781.082789:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3716 00000100:00080000:3.0:1789763781.082801:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763781.082808:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763781.082850:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565907840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763781.082875:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763781.082878:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763781.082879:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763781.082882:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3716 00000100:00080000:3.0:1789763781.082884:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763781.082886:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763781.082895:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763781.082904:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565907968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763781.082910:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763781.082912:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763781.082914:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763781.082916:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3716 00000100:00080000:3.0:1789763781.082918:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763781.082920:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763781.082923:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763781.082929:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565908096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763781.082936:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763781.082939:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763781.082940:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763781.082944:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3716 00000100:00080000:3.0:1789763781.082946:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763781.082948:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763781.082951:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763781.082961:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565908224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763781.082962:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565907840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763781.083071:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565907968:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763781.083093:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565908096:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763781.083121:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565908224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763781.146838:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763782.170794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763783.194852:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789763784.154866:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789763784.154901:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789763784.154914:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763784.154943:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763784.154949:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763784.154956:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789763784.154979:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763781/real 1789763784] req@ffff93f844d65500 x1876699565908224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763784.155000:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565908224/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763784.155008:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763784.155012:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763784.155014:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565908224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763784.155021:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763781/real 1789763784] req@ffff93f844d66a00 x1876699565907840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763784.155029:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565907840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763784.155036:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763784.155038:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763784.155040:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565907840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763784.155044:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763781/real 1789763784] req@ffff93f844d67480 x1876699565907968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763784.155051:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565907968/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763784.155056:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763784.155058:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763784.155060:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565907968:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763784.155063:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763781/real 1789763784] req@ffff93f844d66d80 x1876699565908096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763784.155077:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565908096/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763784.155086:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763784.155089:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763784.155092:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565908096:192.168.203.147@tcp:8:kworker.0 00010000:00010000:3.0:1789763784.218793:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763785.242799:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763786.202752:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763786.202759:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763786.202763:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763786.202771:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3721 00000100:00080000:3.0:1789763786.202780:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763786.202787:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763786.202817:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565908352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763786.202838:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763786.202842:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763786.202843:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763786.202847:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3721 00000100:00080000:3.0:1789763786.202851:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763786.202855:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763786.202862:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763786.202874:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565908480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763786.202881:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763786.202883:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763786.202884:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763786.202887:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3721 00000100:00080000:3.0:1789763786.202889:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763786.202892:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763786.202895:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763786.202906:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565908352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763786.202906:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565908608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763786.202921:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763786.202924:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763786.202925:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763786.202928:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3721 00000100:00080000:3.0:1789763786.202930:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763786.202933:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763786.202936:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763786.202946:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565908736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763786.202974:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763786.203026:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763786/real 1789763786] req@ffff93f844d64380 x1876699565908352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763786.203044:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565908352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763786.203066:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763786.203070:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763786.203075:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565908352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763786.203083:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565908480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763786.203106:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763786.203133:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763786/real 1789763786] req@ffff93f844d64000 x1876699565908480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763786.203147:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565908480/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763786.203156:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763786.203159:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763786.203163:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565908480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763786.203187:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565908608:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763786.203209:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763786.203221:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763786/real 1789763786] req@ffff93f844d64a80 x1876699565908608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763786.203232:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565908608/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763786.203247:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763786.203251:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763786.203255:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565908608:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763786.203261:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565908736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763786.203280:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763786.203291:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763786/real 1789763786] req@ffff93f844d65f80 x1876699565908736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763786.203304:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565908736/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763786.203312:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763786.203315:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763786.203319:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565908736:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763786.266776:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763787.290834:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763788.314754:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763789.338763:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763790.362775:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763791.322728:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763791.322740:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763791.322744:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763791.322754:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3726 00000100:00080000:3.0:1789763791.322763:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763791.322771:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763791.322803:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565908864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763791.322830:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763791.322834:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763791.322836:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763791.322844:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3726 00000100:00080000:3.0:1789763791.322848:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763791.322852:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763791.322862:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763791.322875:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565908992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763791.322887:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763791.322891:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763791.322896:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789763791.322897:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565908864:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763791.322903:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3726 00000100:00080000:3.0:1789763791.322909:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763791.322912:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763791.322917:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763791.322930:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565909120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763791.322952:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1789763791.322958:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763791.322959:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763791.322964:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763791.322969:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3726 00000100:00080000:3.0:1789763791.322975:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763791.322979:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763791.322985:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763791.322998:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565909248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763791.323005:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763791/real 1789763791] req@ffff93f844d65180 x1876699565908864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763791.323028:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565908864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763791.323037:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763791.323040:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763791.323044:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565908864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763791.323063:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565908992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763791.323079:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763791.323104:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763791/real 1789763791] req@ffff93f844d66300 x1876699565908992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763791.323112:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565908992/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763791.323118:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763791.323120:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763791.323122:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565908992:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763791.323126:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565909120:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763791.323140:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763791.323146:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763791/real 1789763791] req@ffff93f844d65f80 x1876699565909120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763791.323154:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565909120/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763791.323159:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763791.323161:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763791.323162:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565909120:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763791.323166:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565909248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763791.323176:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763791.323182:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763791/real 1789763791] req@ffff93f844d64a80 x1876699565909248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763791.323190:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565909248/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763791.323195:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763791.323196:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763791.323198:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565909248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763791.386802:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763792.410757:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763793.434761:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763794.458784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763795.482786:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763796.442794:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763796.442804:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763796.442809:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763796.442819:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3731 00000100:00080000:0.0:1789763796.442831:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763796.442838:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763796.442882:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565909376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763796.442909:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763796.442912:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763796.442914:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763796.442918:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3731 00000100:00080000:0.0:1789763796.442920:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763796.442923:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763796.442933:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763796.442942:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565909504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763796.442954:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565909376:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763796.442956:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763796.442962:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763796.442965:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763796.442969:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3731 00000100:00080000:0.0:1789763796.442971:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763796.442973:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763796.442977:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763796.442993:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565909632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763796.443000:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763796.443003:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763796.443004:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763796.443006:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3731 00000100:00080000:0.0:1789763796.443009:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763796.443011:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763796.443014:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763796.443022:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565909760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763796.443065:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763796.443115:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763796/real 1789763796] req@ffff93f850d41f80 x1876699565909376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763796.443136:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565909376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763796.443146:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763796.443149:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763796.443152:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565909376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763796.443176:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565909504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763796.443192:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763796.443201:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763796/real 1789763796] req@ffff93f850d43800 x1876699565909504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763796.443212:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565909504/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763796.443220:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763796.443224:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763796.443228:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565909504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763796.443234:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565909632:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763796.443256:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763796.443270:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763796/real 1789763796] req@ffff93f850d41500 x1876699565909632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763796.443283:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565909632/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763796.443297:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763796.443300:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763796.443304:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565909632:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763796.443311:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565909760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763796.443335:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763796.443348:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763796/real 1789763796] req@ffff93f850d43480 x1876699565909760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763796.443361:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565909760/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763796.443369:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763796.443372:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763796.443376:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565909760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763796.506784:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763797.530773:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763798.554773:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763799.578802:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763800.602816:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763801.562755:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763801.562767:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763801.562772:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763801.562781:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3736 00000100:00080000:0.0:1789763801.562792:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763801.562799:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763801.562838:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565909888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763801.562859:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763801.562862:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763801.562864:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763801.562867:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3736 00000100:00080000:0.0:1789763801.562871:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763801.562874:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763801.562881:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763801.562887:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565909888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763801.562893:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565910016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763801.562901:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763801.562903:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763801.562905:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763801.562908:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3736 00000100:00080000:0.0:1789763801.562910:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763801.562913:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763801.562916:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763801.562924:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565910144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763801.562931:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763801.562934:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763801.562936:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763801.562948:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3736 00000100:00080000:0.0:1789763801.562951:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763801.562954:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763801.562956:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789763801.562962:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763801.562965:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565910272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763801.563020:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763801/real 1789763801] req@ffff93f850d43100 x1876699565909888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763801.563047:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565909888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763801.563055:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763801.563058:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763801.563061:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565909888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763801.563079:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565910016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763801.563098:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763801.563107:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763801/real 1789763801] req@ffff93f850d41180 x1876699565910016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763801.563116:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565910016/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763801.563121:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763801.563124:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763801.563126:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565910016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763801.563130:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565910144:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763801.563140:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763801.563146:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763801/real 1789763801] req@ffff93f850d41c00 x1876699565910144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763801.563154:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565910144/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763801.563160:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763801.563162:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763801.563164:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565910144:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763801.563168:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565910272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763801.563179:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763801.563185:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763801/real 1789763801] req@ffff93f850d42680 x1876699565910272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763801.563193:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565910272/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763801.563198:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763801.563199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763801.563201:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565910272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763801.626816:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763802.650814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763803.674853:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763804.698799:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763805.722791:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763806.682774:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763806.682786:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763806.682791:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763806.682804:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3741 00000100:00080000:0.0:1789763806.682815:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763806.682822:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763806.682865:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565910400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763806.682894:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763806.682897:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763806.682899:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763806.682906:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3741 00000100:00080000:0.0:1789763806.682909:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763806.682912:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763806.682921:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763806.682930:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565910528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763806.682939:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763806.682942:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763806.682943:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763806.682946:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3741 00000100:00080000:0.0:1789763806.682949:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1789763806.682954:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565910400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763806.682955:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763806.682962:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763806.682984:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565910656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763806.682993:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763806.682996:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763806.682997:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763806.683000:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3741 00000100:00080000:0.0:1789763806.683004:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763806.683007:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763806.683011:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763806.683020:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565910784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763806.683045:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763806.683095:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763806/real 1789763806] req@ffff93f850d40e00 x1876699565910400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763806.683109:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565910400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763806.683119:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763806.683122:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763806.683125:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565910400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763806.683131:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565910528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763806.683147:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763806.683155:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763806/real 1789763806] req@ffff93f850d40000 x1876699565910528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763806.683163:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565910528/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763806.683168:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763806.683170:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763806.683172:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565910528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763806.683194:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565910656:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763806.683207:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763806.683213:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763806/real 1789763806] req@ffff93f850d42300 x1876699565910656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763806.683220:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565910656/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763806.683226:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763806.683227:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763806.683229:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565910656:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763806.683233:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565910784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763806.683242:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763806.683248:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763806/real 1789763806] req@ffff93f850d40380 x1876699565910784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763806.683255:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565910784/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763806.683260:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763806.683262:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763806.683264:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565910784:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763806.746810:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763807.770785:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763808.794814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763809.818790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763810.842895:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763811.802814:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763811.802826:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763811.802832:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763811.802847:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3746 00000100:00080000:3.0:1789763811.802859:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763811.802868:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763811.802916:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565910912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763811.802964:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763811.802983:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763811.802985:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763811.802989:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3746 00000100:00080000:3.0:1789763811.802992:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763811.802997:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763811.803011:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763811.803024:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565911040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763811.803035:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763811.803038:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763811.803040:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763811.803045:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3746 00000100:00080000:3.0:1789763811.803048:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763811.803058:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763811.803064:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565910912:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789763811.803065:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763811.803093:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565911168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763811.803103:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763811.803107:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763811.803109:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763811.803113:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3746 00000100:00080000:3.0:1789763811.803118:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763811.803122:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763811.803126:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763811.803135:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565911296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763811.803182:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763811.803252:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763811/real 1789763811] req@ffff93f844d64000 x1876699565910912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763811.803287:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565910912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763811.803302:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763811.803308:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763811.803314:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565910912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763811.803346:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565911040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763811.803374:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763811.803389:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763811/real 1789763811] req@ffff93f844d64380 x1876699565911040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763811.803411:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565911040/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763811.803427:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763811.803433:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763811.803440:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565911040:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763811.803452:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565911168:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763811.803491:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763811.803510:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763811/real 1789763811] req@ffff93f844d66d80 x1876699565911168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763811.803525:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565911168/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763811.803542:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763811.803546:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763811.803550:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565911168:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763811.803557:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565911296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763811.803600:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763811.803620:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763811/real 1789763811] req@ffff93f844d67480 x1876699565911296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763811.803682:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565911296/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763811.803701:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763811.803705:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763811.803709:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565911296:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763811.866890:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763812.890882:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763813.914746:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763814.938870:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763815.962790:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763816.922769:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763816.922783:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763816.922790:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763816.922804:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3751 00000100:00080000:3.0:1789763816.922830:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763816.922841:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763816.922939:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565911424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763816.922984:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763816.922989:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763816.922991:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763816.922996:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3751 00000100:00080000:3.0:1789763816.923001:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763816.923005:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763816.923018:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763816.923034:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565911552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763816.923048:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763816.923052:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763816.923054:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1789763816.923064:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565911424:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763816.923064:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3751 00000100:00080000:3.0:1789763816.923075:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763816.923081:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763816.923086:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763816.923099:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565911680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763816.923111:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763816.923115:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763816.923117:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763816.923121:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3751 00000100:00080000:3.0:1789763816.923124:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763816.923128:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763816.923133:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763816.923144:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565911808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763816.923170:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763816.923242:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763816/real 1789763816] req@ffff93f844d66a00 x1876699565911424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763816.923269:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565911424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763816.923283:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763816.923288:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763816.923293:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565911424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763816.923327:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565911552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763816.923354:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763816.923368:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763816/real 1789763816] req@ffff93f844d65500 x1876699565911552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763816.923382:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565911552/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763816.923391:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763816.923394:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763816.923398:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565911552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763816.923404:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565911680:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763816.923423:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763816.923433:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763816/real 1789763816] req@ffff93f844d64e00 x1876699565911680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763816.923446:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565911680/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763816.923454:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763816.923457:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763816.923460:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565911680:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763816.923466:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565911808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763816.923482:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763816.923492:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763816/real 1789763816] req@ffff93f844d65c00 x1876699565911808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763816.923505:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565911808/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763816.923514:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763816.923517:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763816.923520:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565911808:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763816.986880:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763818.010982:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763819.035029:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763820.058841:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763821.082943:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763822.042765:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763822.042775:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763822.042779:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763822.042789:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3757 00000100:00080000:3.0:1789763822.042798:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763822.042805:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763822.042842:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565911936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763822.042880:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763822.042884:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763822.042885:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763822.042889:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3757 00000100:00080000:3.0:1789763822.042891:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763822.042894:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763822.042903:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763822.042913:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565912064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763822.042922:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763822.042925:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763822.042926:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763822.042930:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3757 00000100:00080000:3.0:1789763822.042933:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763822.042936:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763822.042940:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763822.042943:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565911936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763822.042951:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565912192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763822.042961:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763822.042963:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763822.042965:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763822.042968:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3757 00000100:00080000:3.0:1789763822.042970:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763822.042973:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763822.042976:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763822.042985:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565912320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789763822.043011:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763822.043066:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763822/real 1789763822] req@ffff93f844d67100 x1876699565911936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763822.043088:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565911936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763822.043102:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763822.043107:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763822.043112:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565911936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763822.043121:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565912064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763822.043163:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763822.043184:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763822/real 1789763822] req@ffff93f844d67800 x1876699565912064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763822.043200:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565912064/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763822.043217:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763822.043221:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763822.043225:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565912064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763822.043252:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565912192:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763822.043282:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763822.043305:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763822/real 1789763822] req@ffff93f844d64700 x1876699565912192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763822.043325:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565912192/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763822.043335:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763822.043338:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763822.043342:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565912192:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763822.043349:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565912320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763822.043375:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763822.043388:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763822/real 1789763822] req@ffff93f844d67b80 x1876699565912320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763822.043403:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565912320/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763822.043413:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763822.043417:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763822.043423:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565912320:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763822.106793:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763823.130830:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763824.154816:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763825.178851:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763826.202812:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763827.162801:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763827.162837:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763827.162847:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763827.162864:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3762 00000100:00080000:3.0:1789763827.162880:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763827.162889:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763827.162942:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565912448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763827.162973:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763827.162977:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763827.162979:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763827.162984:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3762 00000100:00080000:3.0:1789763827.162987:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763827.162992:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763827.163005:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763827.163022:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565912576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763827.163026:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565912448:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763827.163042:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763827.163046:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763827.163049:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763827.163054:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3762 00000100:00080000:3.0:1789763827.163059:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763827.163064:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763827.163070:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763827.163087:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565912704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763827.163100:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763827.163104:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763827.163106:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763827.163111:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3762 00000800:00000100:0.0:1789763827.163114:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763827.163118:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763827.163125:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763827.163132:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763827.163145:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565912832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789763827.163173:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763827/real 1789763827] req@ffff93f844d65880 x1876699565912448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763827.163197:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565912448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763827.163211:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763827.163215:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763827.163218:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565912448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763827.163243:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565912576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763827.163273:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763827.163284:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763827/real 1789763827] req@ffff93f844d66680 x1876699565912576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763827.163293:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565912576/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763827.163300:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763827.163302:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763827.163305:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565912576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763827.163309:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565912704:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789763827.163323:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763827.163330:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763827/real 1789763827] req@ffff93f844d67b80 x1876699565912704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763827.163338:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565912704/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763827.163350:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763827.163352:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763827.163356:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565912704:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763827.163361:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565912832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789763827.163372:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763827.163380:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763827/real 1789763827] req@ffff93f844d64700 x1876699565912832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763827.163392:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565912832/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763827.163402:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763827.163406:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763827.163409:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565912832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763827.226861:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763828.250831:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763829.274889:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763830.298792:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763831.322837:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763832.282850:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763832.282861:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763832.282866:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763832.282878:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3767 00000100:00080000:0.0:1789763832.282888:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763832.282895:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763832.282936:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565912960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763832.282975:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763832.282978:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763832.282980:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763832.282984:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3767 00000100:00080000:0.0:1789763832.282988:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763832.282992:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763832.283001:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763832.283016:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565913088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763832.283023:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565912960:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763832.283045:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763832.283049:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763832.283051:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763832.283055:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3767 00000100:00080000:0.0:1789763832.283060:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763832.283063:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763832.283068:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763832.283081:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565913216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763832.283090:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763832.283094:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763832.283096:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763832.283101:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3767 00000100:00080000:0.0:1789763832.283105:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763832.283109:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:3.0:1789763832.283115:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1789763832.283118:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763832.283132:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565913344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763832.283171:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763832/real 1789763832] req@ffff93f850d40700 x1876699565912960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763832.283190:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565912960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763832.283201:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763832.283205:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763832.283209:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565912960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763832.283234:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565913088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763832.283266:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763832.283277:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763832/real 1789763832] req@ffff93f850d43b80 x1876699565913088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763832.283288:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565913088/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763832.283296:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763832.283299:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763832.283302:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565913088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763832.283307:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565913216:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763832.283325:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763832.283333:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763832/real 1789763832] req@ffff93f850d40380 x1876699565913216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763832.283344:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565913216/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763832.283351:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763832.283353:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763832.283356:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565913216:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763832.283360:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565913344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763832.283375:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763832.283382:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763832/real 1789763832] req@ffff93f850d42300 x1876699565913344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763832.283392:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565913344/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763832.283399:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763832.283402:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763832.283404:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565913344:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763832.346836:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763833.370879:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763834.394795:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763835.418922:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763836.442824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763837.402771:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763837.402795:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763837.402801:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763837.402815:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3772 00000100:00080000:0.0:1789763837.402828:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763837.402837:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763837.402912:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565913472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763837.402946:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763837.402950:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763837.402953:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763837.402957:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3772 00000100:00080000:0.0:1789763837.402962:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763837.402967:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763837.402980:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763837.402994:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565913600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763837.403020:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565913472:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763837.403031:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763837.403037:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763837.403039:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763837.403044:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3772 00000100:00080000:0.0:1789763837.403048:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763837.403053:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763837.403071:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763837.403084:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565913728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763837.403096:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763837.403101:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763837.403103:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763837.403108:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3772 00000100:00080000:0.0:1789763837.403111:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763837.403115:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763837.403120:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763837.403164:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565913856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763837.403181:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763837.403275:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763837/real 1789763837] req@ffff93f850d40000 x1876699565913472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763837.403318:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565913472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763837.403332:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763837.403337:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763837.403341:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565913472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763837.403370:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565913600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763837.403397:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763837.403410:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763837/real 1789763837] req@ffff93f850d40e00 x1876699565913600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763837.403422:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565913600/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763837.403430:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763837.403433:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763837.403436:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565913600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763837.403441:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565913728:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763837.403458:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763837.403466:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763837/real 1789763837] req@ffff93f850d42680 x1876699565913728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763837.403493:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565913728/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763837.403501:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763837.403504:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763837.403507:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565913728:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763837.403512:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565913856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763837.403528:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763837.403536:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763837/real 1789763837] req@ffff93f850d41c00 x1876699565913856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763837.403546:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565913856/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763837.403553:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763837.403556:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763837.403558:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565913856:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763837.466862:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763838.490884:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763839.514823:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763840.538861:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763841.562850:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763842.522856:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763842.522873:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763842.522879:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763842.522894:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3777 00000100:00080000:2.0:1789763842.522908:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763842.522917:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763842.522970:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565913984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763842.523015:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763842.523028:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763842.523032:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763842.523042:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3777 00000100:00080000:2.0:1789763842.523051:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763842.523061:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763842.523079:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763842.523098:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565913984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763842.523121:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565914112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763842.523145:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763842.523151:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763842.523154:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763842.523160:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3777 00000100:00080000:2.0:1789763842.523168:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763842.523191:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763842.523201:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763842.523218:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565914240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763842.523230:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763842.523235:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763842.523237:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763842.523241:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3777 00000100:00080000:2.0:1789763842.523246:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1789763842.523259:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763842.523266:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763842.523276:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763842.523308:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565914368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763842.523385:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763842/real 1789763842] req@ffff93f8517c2300 x1876699565913984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763842.523419:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565913984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763842.523437:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763842.523443:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763842.523449:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565913984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763842.523491:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565914112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763842.523530:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763842.523553:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763842/real 1789763842] req@ffff93f8517c1f80 x1876699565914112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763842.523578:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565914112/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763842.523596:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763842.523602:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763842.523610:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565914112:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763842.523623:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565914240:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763842.523700:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763842.523755:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763842/real 1789763842] req@ffff93f8517c0000 x1876699565914240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763842.523781:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565914240/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763842.523812:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763842.523821:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763842.523829:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565914240:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763842.523839:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565914368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763842.523874:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763842.523889:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763842/real 1789763842] req@ffff93f8517c3b80 x1876699565914368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763842.523904:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565914368/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763842.523914:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763842.523918:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763842.523922:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565914368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763842.586885:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763843.610891:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763844.634910:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763845.658889:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763846.682777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763847.642772:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763847.642786:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763847.642793:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763847.642808:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3782 00000100:00080000:2.0:1789763847.642840:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763847.642851:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763847.642895:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565914496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763847.642922:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763847.642926:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763847.642928:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763847.642932:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3782 00000100:00080000:2.0:1789763847.642936:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763847.642949:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763847.642960:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763847.642970:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565914496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763847.642984:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565914624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763847.643003:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763847.643009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763847.643012:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763847.643033:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3782 00000100:00080000:2.0:1789763847.643041:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763847.643046:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763847.643052:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763847.643062:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565914752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763847.643070:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763847.643073:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763847.643074:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763847.643077:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3782 00000100:00080000:2.0:1789763847.643080:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763847.643102:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763847.643118:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565914624:192.168.203.147@tcp:38:kworker.0 00000008:00100000:2.0:1789763847.643125:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763847.643171:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565914880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763847.643186:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565914752:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763847.643239:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565914880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763847.706810:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763848.730788:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763849.754874:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789763850.715068:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789763850.715137:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789763850.715161:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763850.715225:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763850.715248:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763850.715267:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789763850.715281:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763847/real 1789763850] req@ffff93f8517c2680 x1876699565914624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763850.715337:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565914624/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763850.715364:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763850.715377:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763850.715390:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565914624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789763850.715408:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763847/real 1789763850] req@ffff93f8517c0380 x1876699565914752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763850.715438:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565914752/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763850.715454:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763850.715463:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763850.715470:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565914752:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789763850.715484:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763847/real 1789763850] req@ffff93f8517c1500 x1876699565914496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763850.715509:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565914496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763850.715525:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763850.715531:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763850.715538:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565914496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763850.715579:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763847/real 1789763850] req@ffff93f8517c0a80 x1876699565914880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763850.715605:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565914880/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763850.715622:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789763850.715628:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789763850.715636:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565914880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763850.778861:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763851.802906:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763852.762891:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763852.762903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763852.762908:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763852.762920:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3787 00000100:00080000:0.0:1789763852.762932:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763852.762941:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763852.762985:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565915008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763852.763029:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763852.763036:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763852.763039:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763852.763047:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3787 00000100:00080000:0.0:1789763852.763055:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763852.763062:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763852.763078:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763852.763103:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565915136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763852.763118:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565915008:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763852.763123:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763852.763130:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763852.763133:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763852.763139:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3787 00000100:00080000:0.0:1789763852.763145:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763852.763152:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763852.763161:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763852.763179:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565915264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763852.763196:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763852.763203:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763852.763206:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1789763852.763209:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763852.763213:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3787 00000100:00080000:0.0:1789763852.763219:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763852.763227:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763852.763249:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763852.763271:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565915392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763852.763278:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763852/real 1789763852] req@ffff93f850d41180 x1876699565915008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763852.763336:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565915008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763852.763358:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763852.763369:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763852.763376:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565915008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763852.763419:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565915136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763852.763470:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763852.763505:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763852/real 1789763852] req@ffff93f850d43100 x1876699565915136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763852.763547:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565915136/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763852.763565:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763852.763571:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763852.763578:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565915136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763852.763590:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565915264:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763852.763714:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763852.763744:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763852/real 1789763852] req@ffff93f850d43480 x1876699565915264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763852.763770:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565915264/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763852.763786:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763852.763792:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763852.763798:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565915264:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763852.763809:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565915392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763852.763852:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763852.763875:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763852/real 1789763852] req@ffff93f850d41500 x1876699565915392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763852.763898:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565915392/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763852.764210:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763852.764218:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763852.764226:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565915392:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763852.826889:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763853.850965:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763854.874805:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763855.898891:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763856.922909:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763857.882811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763857.882824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763857.882831:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763857.882844:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3792 00000100:00080000:0.0:1789763857.882857:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763857.882867:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763857.882917:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565915520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763857.882961:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763857.882966:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763857.882970:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763857.882979:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3792 00000100:00080000:0.0:1789763857.882986:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763857.882993:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763857.883007:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763857.883029:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565915520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763857.883044:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565915648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763857.883072:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763857.883080:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763857.883085:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763857.883095:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3792 00000100:00080000:0.0:1789763857.883122:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789763857.883123:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763857.883139:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763857.883150:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763857.883176:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565915776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763857.883192:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763857/real 1789763857] req@ffff93f850d43800 x1876699565915520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763857.883194:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763857.883210:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763857.883217:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763857.883233:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3792 00000100:00080000:0.0:1789763857.883240:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763857.883251:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763857.883253:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565915520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1789763857.883257:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763857.883283:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565915904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763857.883291:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763857.883323:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763857.883334:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565915520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763857.883389:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565915648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763857.883439:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763857.883470:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763857/real 1789763857] req@ffff93f850d41f80 x1876699565915648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763857.883498:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565915648/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763857.883516:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763857.883523:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763857.883530:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565915648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763857.883544:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565915776:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763857.883587:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763857.883698:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763857/real 1789763857] req@ffff93f850d41500 x1876699565915776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763857.883721:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565915776/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763857.883734:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763857.883740:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763857.883746:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565915776:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763857.883758:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565915904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763857.883797:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763857.883826:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763857/real 1789763857] req@ffff93f850d43480 x1876699565915904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763857.883855:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565915904/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763857.883866:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763857.883871:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763857.883876:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565915904:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763857.946870:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763858.970800:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763859.994865:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763861.018844:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763862.042872:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763863.002744:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763863.002753:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763863.002765:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763863.002776:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3797 00000100:00080000:2.0:1789763863.002784:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763863.002792:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763863.002828:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565916032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763863.002855:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763863.002858:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763863.002860:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763863.002863:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3797 00000100:00080000:2.0:1789763863.002866:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763863.002870:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763863.002879:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763863.002896:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565916160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763863.002896:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565916032:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763863.002920:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763863.002923:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763863.002924:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763863.002927:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3797 00000100:00080000:2.0:1789763863.002930:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763863.002933:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763863.002938:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763863.002948:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565916288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763863.002956:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763863.002959:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763863.002961:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763863.002964:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3797 00000100:00080000:2.0:1789763863.002967:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763863.002970:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763863.002973:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789763863.002978:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763863.002984:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565916416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763863.003033:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763863/real 1789763863] req@ffff93f8517c2d80 x1876699565916032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763863.003055:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565916032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763863.003064:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763863.003067:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763863.003071:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565916032:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763863.003094:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565916160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763863.003115:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763863.003126:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763863/real 1789763863] req@ffff93f8517c2a00 x1876699565916160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763863.003136:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565916160/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763863.003144:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763863.003146:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763863.003149:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565916160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763863.003154:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565916288:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763863.003167:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763863.003174:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763863/real 1789763863] req@ffff93f8517c1180 x1876699565916288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763863.003184:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565916288/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763863.003191:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763863.003193:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763863.003203:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565916288:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763863.003208:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565916416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763863.003222:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763863.003230:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763863/real 1789763863] req@ffff93f8517c3100 x1876699565916416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763863.003239:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565916416/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763863.003247:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763863.003250:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763863.003252:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565916416:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763863.066791:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763864.090865:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763865.114743:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763866.138780:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763867.162928:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763868.122817:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763868.122835:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763868.122842:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763868.122859:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3803 00000100:00080000:2.0:1789763868.122877:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763868.122889:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763868.122938:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565916544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763868.122990:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763868.122998:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763868.123002:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763868.123010:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3803 00000100:00080000:2.0:1789763868.123019:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763868.123028:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763868.123046:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763868.123076:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565916672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763868.123090:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565916544:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789763868.123100:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763868.123107:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763868.123110:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763868.123117:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3803 00000100:00080000:2.0:1789763868.123124:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763868.123131:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763868.123139:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763868.123155:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565916800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763868.123170:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763868.123176:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763868.123178:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763868.123184:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3803 00000100:00080000:2.0:1789763868.123202:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763868.123208:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763868.123215:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789763868.123217:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763868.123245:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565916928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763868.123326:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763868/real 1789763868] req@ffff93f8517c1880 x1876699565916544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763868.123373:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565916544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763868.123393:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763868.123400:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763868.123406:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565916544:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763868.123449:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565916672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763868.123487:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763868.123505:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763868/real 1789763868] req@ffff93f8517c3480 x1876699565916672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763868.123554:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565916672/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763868.123567:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763868.123571:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763868.123576:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565916672:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763868.123584:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565916800:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763868.123616:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763868.123630:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763868/real 1789763868] req@ffff93f8517c3800 x1876699565916800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763868.123678:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565916800/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763868.123691:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763868.123695:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763868.123700:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565916800:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763868.123708:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565916928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763868.123735:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763868.123749:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763868/real 1789763868] req@ffff93f8517c0e00 x1876699565916928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763868.123764:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565916928/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763868.123774:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763868.123778:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763868.123783:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565916928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763868.186846:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763869.210865:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763870.234866:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763871.258875:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763872.282745:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763873.242823:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763873.242831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763873.242835:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763873.242843:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3808 00000100:00080000:0.0:1789763873.242851:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763873.242857:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763873.242888:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565917056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763873.242910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763873.242913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763873.242914:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763873.242917:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3808 00000100:00080000:0.0:1789763873.242920:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763873.242923:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763873.242930:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763873.242940:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565917184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763873.242945:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565917056:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763873.242952:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763873.242955:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763873.242956:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763873.242959:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3808 00000100:00080000:0.0:1789763873.242961:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763873.242964:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763873.242967:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763873.242976:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565917312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763873.242983:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763873.242987:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763873.242989:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763873.242993:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3808 00000100:00080000:0.0:1789763873.242998:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763873.243002:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:3.0:1789763873.243007:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1789763873.243011:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763873.243027:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565917440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763873.243084:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763873/real 1789763873] req@ffff93f850d43100 x1876699565917056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763873.243111:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565917056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763873.243125:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763873.243150:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763873.243158:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565917056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763873.243193:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565917184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763873.243230:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763873.243249:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763873/real 1789763873] req@ffff93f850d41180 x1876699565917184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763873.243267:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565917184/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763873.243292:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763873.243295:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763873.243298:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565917184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763873.243303:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565917312:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763873.243331:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763873.243359:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763873/real 1789763873] req@ffff93f850d41c00 x1876699565917312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763873.243375:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565917312/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763873.243386:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763873.243390:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763873.243395:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565917312:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763873.243413:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565917440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763873.243438:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763873.243451:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763873/real 1789763873] req@ffff93f850d42680 x1876699565917440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763873.243466:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565917440/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763873.243476:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763873.243503:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763873.243508:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565917440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763873.306806:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763874.330861:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763875.354814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763876.378896:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763877.402960:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763878.362800:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763878.362812:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763878.362817:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763878.362829:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3813 00000100:00080000:0.0:1789763878.362858:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763878.362866:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763878.362917:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565917568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763878.362964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763878.362967:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763878.362969:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763878.362974:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3813 00000100:00080000:0.0:1789763878.362977:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763878.362981:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763878.362992:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763878.363003:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565917696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763878.363013:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565917568:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763878.363014:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763878.363023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763878.363026:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763878.363031:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3813 00000100:00080000:0.0:1789763878.363033:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763878.363036:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763878.363039:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763878.363049:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565917824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763878.363057:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763878.363060:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763878.363062:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763878.363065:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3813 00000100:00080000:0.0:1789763878.363068:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763878.363071:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763878.363074:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763878.363095:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565917952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763878.363123:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763878.363169:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763878/real 1789763878] req@ffff93f850d40e00 x1876699565917568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763878.363186:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565917568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763878.363196:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763878.363210:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763878.363230:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565917568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763878.363261:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565917696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763878.363305:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763878.363320:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763878/real 1789763878] req@ffff93f850d40000 x1876699565917696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763878.363334:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565917696/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763878.363362:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763878.363366:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763878.363370:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565917696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763878.363377:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565917824:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763878.363406:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763878.363432:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763878/real 1789763878] req@ffff93f850d42300 x1876699565917824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763878.363462:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565917824/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763878.363470:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763878.363472:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763878.363506:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565917824:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763878.363511:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565917952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763878.363529:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763878.363537:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763878/real 1789763878] req@ffff93f850d40380 x1876699565917952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763878.363565:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565917952/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763878.363572:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763878.363574:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763878.363577:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565917952:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763878.426904:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763879.450914:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763880.474860:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763881.498861:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763882.522885:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763883.482763:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763883.482778:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763883.482784:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763883.482811:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3818 00000100:00080000:0.0:1789763883.482825:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763883.482839:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763883.482891:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565918080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763883.482947:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763883.482955:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763883.482958:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763883.482966:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3818 00000100:00080000:0.0:1789763883.482988:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763883.482997:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763883.483013:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763883.483015:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565918080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763883.483037:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565918208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763883.483058:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763883.483063:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763883.483066:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763883.483090:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3818 00000100:00080000:0.0:1789763883.483095:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763883.483111:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763883.483119:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763883.483137:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565918336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763883.483142:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763883.483154:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763883.483176:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763883.483179:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763883.483185:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3818 00000100:00080000:0.0:1789763883.483192:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763883.483198:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763883.483206:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763883.483225:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565918464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763883.483245:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763883/real 1789763883] req@ffff93f850d43b80 x1876699565918080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763883.483297:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565918080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763883.483318:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763883.483325:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763883.483333:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565918080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763883.483382:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565918208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763883.483442:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763883.483492:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763883/real 1789763883] req@ffff93f850d40700 x1876699565918208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763883.483508:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565918208/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763883.483518:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763883.483522:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763883.483526:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565918208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763883.483533:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565918336:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763883.483557:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763883.483569:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763883/real 1789763883] req@ffff93f850d40380 x1876699565918336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763883.483591:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565918336/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763883.483601:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763883.483604:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763883.483608:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565918336:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763883.483614:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565918464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763883.483632:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763883.483668:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763883/real 1789763883] req@ffff93f850d42300 x1876699565918464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763883.483704:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565918464/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763883.483714:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763883.483718:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763883.483722:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565918464:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763883.546816:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763884.570895:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763885.594850:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763886.618928:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763887.642894:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763888.602859:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763888.602872:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763888.602878:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763888.602890:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3823 00000100:00080000:1.0:1789763888.602902:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763888.602922:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763888.602973:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb800 x1876699565918592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763888.603017:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763888.603022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763888.603024:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763888.603041:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3823 00000100:00080000:1.0:1789763888.603046:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763888.603050:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763888.603062:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763888.603077:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb480 x1876699565918720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763888.603091:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763888.603096:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763888.603099:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763888.603104:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3823 00000100:00080000:1.0:1789763888.603108:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763888.603113:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763888.603119:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:1.0:1789763888.603131:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccaa00 x1876699565918848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763888.603146:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763888.603155:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:3.0:1789763888.603166:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565918592:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1789763888.603168:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763888.603184:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3823 00000100:00080000:1.0:1789763888.603189:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763888.603193:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763888.603199:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763888.603216:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8700 x1876699565918976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763888.603282:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763888.603352:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763888/real 1789763888] req@ffff93f844ccb800 x1876699565918592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763888.603414:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb800 x1876699565918592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763888.603430:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763888.603436:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763888.603441:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565918592:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763888.603452:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565918720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763888.603480:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763888.603496:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763888/real 1789763888] req@ffff93f844ccb480 x1876699565918720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763888.603512:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb480 x1876699565918720/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763888.603522:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763888.603527:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763888.603531:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565918720:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763888.603562:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565918848:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763888.603583:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763888.603595:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763888/real 1789763888] req@ffff93f844ccaa00 x1876699565918848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763888.603610:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccaa00 x1876699565918848/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763888.603629:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763888.603633:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763888.603638:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565918848:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763888.603677:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565918976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763888.603703:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763888.603717:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763888/real 1789763888] req@ffff93f844cc8700 x1876699565918976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763888.603732:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8700 x1876699565918976/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763888.603742:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763888.603746:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763888.603750:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565918976:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763888.666861:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763889.690845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763890.714759:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763891.738936:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763892.762919:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763893.722769:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763893.722780:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763893.722796:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763893.722809:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3828 00000100:00080000:0.0:1789763893.722829:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763893.722841:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763893.722895:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565919104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763893.722921:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763893.722923:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763893.722926:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763893.722934:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3828 00000100:00080000:0.0:1789763893.722939:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763893.722943:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763893.722957:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763893.722963:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565919104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763893.722978:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565919232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763893.722993:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763893.722997:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763893.722999:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763893.723003:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3828 00000100:00080000:0.0:1789763893.723007:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763893.723010:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763893.723015:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763893.723026:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565919360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763893.723035:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763893.723037:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763893.723041:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763893.723042:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763893.723046:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3828 00000100:00080000:0.0:1789763893.723049:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763893.723052:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763893.723055:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763893.723063:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565919488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763893.723095:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763893/real 1789763893] req@ffff93f850d40000 x1876699565919104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763893.723117:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565919104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763893.723126:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763893.723131:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763893.723135:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565919104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763893.723168:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565919232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763893.723196:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763893.723209:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763893/real 1789763893] req@ffff93f850d40e00 x1876699565919232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763893.723219:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565919232/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763893.723226:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763893.723229:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763893.723232:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565919232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763893.723237:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565919360:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763893.723253:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763893.723260:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763893/real 1789763893] req@ffff93f850d42680 x1876699565919360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763893.723279:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565919360/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763893.723286:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763893.723289:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763893.723292:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565919360:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763893.723299:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565919488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763893.723318:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763893.723345:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763893/real 1789763893] req@ffff93f850d41c00 x1876699565919488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763893.723365:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565919488/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763893.723387:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763893.723391:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763893.723396:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565919488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763893.786755:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763894.810831:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763895.834927:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763896.858928:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763897.882914:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763898.842893:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763898.842915:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763898.842924:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763898.842943:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3833 00000100:00080000:2.0:1789763898.842961:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763898.842976:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763898.843050:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0e00 x1876699565919616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763898.843104:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763898.843113:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763898.843117:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763898.843126:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3833 00000100:00080000:2.0:1789763898.843135:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763898.843145:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763898.843165:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763898.843186:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565919616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763898.843199:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3800 x1876699565919744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763898.843226:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763898.843232:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763898.843236:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763898.843260:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3833 00000100:00080000:2.0:1789763898.843273:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763898.843283:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763898.843315:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:3.0:1789763898.843327:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763898.843355:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3480 x1876699565919872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763898.843378:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763898.843385:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763898.843389:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763898.843398:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3833 00000100:00080000:2.0:1789763898.843407:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763898.843418:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763898.843429:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763898.843430:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763898/real 1789763898] req@ffff93f8517c0e00 x1876699565919616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763898.843460:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1880 x1876699565920000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763898.843480:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0e00 x1876699565919616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763898.843503:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763898.843509:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763898.843516:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565919616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763898.843557:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565919744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763898.843594:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763898.843612:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763898/real 1789763898] req@ffff93f8517c3800 x1876699565919744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763898.843628:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3800 x1876699565919744/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763898.843639:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763898.843725:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763898.843731:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565919744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763898.843740:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565919872:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763898.843769:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763898.843785:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763898/real 1789763898] req@ffff93f8517c3480 x1876699565919872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763898.843817:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3480 x1876699565919872/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763898.843828:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763898.843832:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763898.843837:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565919872:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763898.843844:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565920000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763898.843870:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763898.843891:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763898/real 1789763898] req@ffff93f8517c1880 x1876699565920000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763898.843915:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1880 x1876699565920000/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763898.843932:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763898.843938:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763898.843946:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565920000:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763898.906909:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763899.930833:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763900.954807:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763901.978953:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763903.002865:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763903.962872:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763903.962883:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763903.962887:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763903.962898:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3838 00000100:00080000:0.0:1789763903.962912:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763903.962923:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763903.962968:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565920128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763903.963000:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763903.963004:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763903.963006:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763903.963011:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3838 00000100:00080000:0.0:1789763903.963014:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763903.963018:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763903.963027:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763903.963041:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565920256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763903.963053:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763903.963057:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763903.963059:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763903.963092:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3838 00000100:00080000:0.0:1789763903.963097:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1789763903.963102:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565920128:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763903.963108:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763903.963124:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763903.963141:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565920384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763903.963156:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763903.963161:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763903.963164:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763903.963168:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3838 00000100:00080000:0.0:1789763903.963174:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763903.963178:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763903.963187:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789763903.963206:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763903.963214:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565920512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763903.963327:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763903/real 1789763903] req@ffff93f850d41180 x1876699565920128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763903.963367:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565920128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763903.963382:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763903.963389:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763903.963396:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565920128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763903.963408:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565920256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763903.963442:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763903.963458:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763903/real 1789763903] req@ffff93f850d43100 x1876699565920256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763903.963475:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565920256/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763903.963485:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763903.963490:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763903.963494:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565920256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763903.963523:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565920384:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763903.963544:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763903.963556:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763903/real 1789763903] req@ffff93f850d43480 x1876699565920384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763903.963571:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565920384/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763903.963581:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763903.963585:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763903.963588:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565920384:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763903.963595:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565920512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763903.963614:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763903.963625:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763903/real 1789763903] req@ffff93f850d41500 x1876699565920512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763903.963639:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565920512/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763903.963704:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763903.963708:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763903.963713:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565920512:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763904.026959:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763905.050883:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763906.074928:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763907.098798:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763908.122960:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789763909.082825:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763909.082847:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1789763909.082857:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763909.082878:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3844 00000100:00080000:1.0:1789763909.082899:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763909.082911:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1789763909.082996:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844ccb100 x1876699565920640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763909.083042:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763909.083047:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763909.083050:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763909.083056:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3844 00000100:00080000:1.0:1789763909.083060:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763909.083065:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763909.083081:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763909.083097:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cca680 x1876699565920768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763909.083112:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763909.083117:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1789763909.083121:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763909.083126:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3844 00000100:00080000:1.0:1789763909.083130:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763909.083135:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763909.083142:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763909.083151:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565920640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763909.083162:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc8e00 x1876699565920896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1789763909.083180:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1789763909.083185:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1789763909.083187:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1789763909.083192:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3844 00000100:00080000:1.0:1789763909.083196:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1789763909.083201:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1789763909.083206:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1789763909.083250:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844cc9c00 x1876699565921024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763909.083321:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763909.083410:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763909/real 1789763909] req@ffff93f844ccb100 x1876699565920640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763909.083443:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844ccb100 x1876699565920640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763909.083459:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763909.083466:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763909.083472:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565920640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763909.083483:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565920768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763909.083518:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763909.083535:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763909/real 1789763909] req@ffff93f844cca680 x1876699565920768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763909.083551:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cca680 x1876699565920768/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763909.083562:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763909.083567:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763909.083571:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565920768:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763909.083608:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565920896:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763909.083632:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763909.083691:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763909/real 1789763909] req@ffff93f844cc8e00 x1876699565920896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763909.083718:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc8e00 x1876699565920896/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763909.083735:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763909.083742:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763909.083749:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565920896:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763909.083762:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565921024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763909.083803:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763909.083821:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763909/real 1789763909] req@ffff93f844cc9c00 x1876699565921024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763909.083836:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844cc9c00 x1876699565921024/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763909.083847:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763909.083851:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763909.083856:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844cc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565921024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763909.146845:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763910.170923:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763911.194854:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763912.218810:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763913.242925:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763914.202870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763914.202891:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763914.202900:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763914.202921:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3849 00000100:00080000:0.0:1789763914.202940:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763914.202956:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763914.203033:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565921152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763914.203084:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763914.203089:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763914.203093:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763914.203099:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3849 00000100:00080000:0.0:1789763914.203105:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763914.203112:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763914.203127:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763914.203148:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565921280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763914.203161:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763914.203166:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763914.203168:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763914.203173:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3849 00000100:00080000:0.0:1789763914.203204:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763914.203214:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763914.203222:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763914.203240:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565921408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763914.203257:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763914.203266:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763914.203270:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1789763914.203270:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565921152:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763914.203290:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3849 00000100:00080000:0.0:1789763914.203302:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763914.203309:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763914.203317:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763914.203342:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565921536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763914.203460:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565921280:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763914.203517:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565921408:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763914.203545:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565921536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763914.266951:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763915.290827:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763916.314889:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1789763917.275201:0:2347:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1789763917.275302:0:2347:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1789763917.275349:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763917.275445:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763917.275465:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1789763917.275481:0:2347:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789763917.275838:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763914/real 1789763917] req@ffff93f850d41500 x1876699565921408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763917.275893:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565921408/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763917.275928:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763917.275938:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763917.275947:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565921408:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763917.275964:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763914/real 1789763917] req@ffff93f850d43480 x1876699565921536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763917.275989:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565921536/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763917.276003:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763917.276009:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763917.276016:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565921536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763917.276028:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763914/real 1789763917] req@ffff93f850d41f80 x1876699565921152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763917.276049:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565921152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763917.276063:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763917.276068:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763917.276074:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565921152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763917.276086:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763914/real 1789763917] req@ffff93f850d43800 x1876699565921280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763917.276105:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565921280/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763917.276119:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763917.276125:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763917.276131:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565921280:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763917.338899:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763918.362819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763919.322820:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763919.322836:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763919.322841:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763919.322854:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3854 00000100:00080000:0.0:1789763919.322869:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763919.322878:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763919.322930:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565921664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763919.322959:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763919.322965:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763919.322968:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763919.322977:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3854 00000100:00080000:0.0:1789763919.322983:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763919.322989:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763919.323002:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763919.323015:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565921664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763919.323024:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565921792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763919.323068:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763919.323074:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763919.323076:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763919.323083:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3854 00000100:00080000:0.0:1789763919.323090:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763919.323099:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763919.323107:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763919.323157:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565921920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763919.323171:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763919.323182:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000800:00000100:2.0:1789763919.323182:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763919.323187:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763919.323199:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3854 00000100:00080000:0.0:1789763919.323207:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763919.323214:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763919.323222:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763919.323237:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565922048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763919.323304:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763919/real 1789763919] req@ffff93f850d43100 x1876699565921664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763919.323342:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565921664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763919.323363:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763919.323402:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763919.323411:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565921664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763919.323463:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565921792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763919.323515:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763919.323542:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763919/real 1789763919] req@ffff93f850d41180 x1876699565921792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763919.323568:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565921792/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763919.323585:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763919.323593:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763919.323599:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565921792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763919.323613:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565921920:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763919.323703:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763919.323734:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763919/real 1789763919] req@ffff93f850d41c00 x1876699565921920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763919.323759:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565921920/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763919.323777:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763919.323784:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763919.323791:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565921920:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763919.323805:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565922048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763919.323849:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763919.323874:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763919/real 1789763919] req@ffff93f850d42680 x1876699565922048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763919.323899:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565922048/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763919.323916:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763919.323922:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763919.323931:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565922048:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763919.386956:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763920.410785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763921.434913:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763922.458867:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763923.482854:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763924.442806:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763924.442828:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763924.442836:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763924.442866:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3859 00000100:00080000:0.0:1789763924.442885:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763924.442918:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763924.442984:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565922176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763924.443036:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763924.443042:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763924.443045:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763924.443051:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3859 00000100:00080000:0.0:1789763924.443058:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763924.443066:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763924.443083:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763924.443100:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565922176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763924.443137:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565922304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763924.443161:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763924.443166:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763924.443169:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763924.443175:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3859 00000100:00080000:0.0:1789763924.443181:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763924.443186:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763924.443195:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763924.443215:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565922432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763924.443230:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763924.443235:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763924.443237:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763924.443242:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3859 00000100:00080000:0.0:1789763924.443247:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1789763924.443254:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763924.443261:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763924.443270:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763924.443289:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565922560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763924.443332:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763924/real 1789763924] req@ffff93f850d40e00 x1876699565922176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763924.443362:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565922176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763924.443378:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763924.443383:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763924.443390:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565922176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763924.443424:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565922304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763924.443463:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763924.443479:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763924/real 1789763924] req@ffff93f850d40000 x1876699565922304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763924.443496:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565922304/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763924.443506:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763924.443511:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763924.443515:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565922304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763924.443522:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565922432:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763924.443543:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763924.443554:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763924/real 1789763924] req@ffff93f850d42300 x1876699565922432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763924.443569:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565922432/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763924.443587:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763924.443591:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763924.443595:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565922432:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763924.443602:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565922560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763924.443632:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763924.443703:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763924/real 1789763924] req@ffff93f850d40380 x1876699565922560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763924.443728:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565922560/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763924.443739:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763924.443743:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763924.443748:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565922560:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763924.506853:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763925.530844:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763926.554883:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763927.578873:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763928.602879:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763929.562828:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763929.562849:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763929.562858:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763929.562896:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3864 00000100:00080000:2.0:1789763929.562916:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763929.562946:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763929.563026:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3100 x1876699565922688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763929.563118:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763929.563126:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763929.563147:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763929.563157:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3864 00000100:00080000:2.0:1789763929.563166:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763929.563178:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763929.563201:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763929.563218:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565922688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763929.563234:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1180 x1876699565922816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763929.563254:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763929.563259:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763929.563261:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763929.563267:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3864 00000100:00080000:2.0:1789763929.563273:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763929.563278:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763929.563285:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763929.563303:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2a00 x1876699565922944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763929.563317:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763929.563323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763929.563326:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763929.563330:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3864 00000100:00080000:2.0:1789763929.563335:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763929.563339:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763929.563350:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1789763929.563352:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763929.563369:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2d80 x1876699565923072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763929.563452:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763929/real 1789763929] req@ffff93f8517c3100 x1876699565922688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763929.563488:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3100 x1876699565922688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763929.563506:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763929.563512:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763929.563520:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565922688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763929.563562:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565922816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763929.563603:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763929.563623:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763929/real 1789763929] req@ffff93f8517c1180 x1876699565922816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763929.563720:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1180 x1876699565922816/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763929.563741:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763929.563750:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763929.563757:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565922816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763929.563772:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565922944:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763929.563814:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763929.563832:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763929/real 1789763929] req@ffff93f8517c2a00 x1876699565922944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763929.563847:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2a00 x1876699565922944/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763929.563858:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763929.563863:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763929.563868:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565922944:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763929.563970:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565923072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763929.564002:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763929.564229:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763929/real 1789763929] req@ffff93f8517c2d80 x1876699565923072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763929.564250:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2d80 x1876699565923072/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763929.564261:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763929.564266:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763929.564272:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565923072:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763929.626868:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763930.650972:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763931.674882:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763932.698876:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763933.722901:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763934.682859:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763934.682875:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763934.682883:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763934.682900:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3869 00000100:00080000:2.0:1789763934.682916:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763934.682929:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763934.683008:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565923200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763934.683096:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763934.683102:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763934.683106:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763934.683116:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3869 00000100:00080000:2.0:1789763934.683124:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763934.683142:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763934.683159:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763934.683161:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565923200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763934.683187:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565923328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763934.683205:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763934.683210:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763934.683213:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763934.683219:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3869 00000100:00080000:2.0:1789763934.683225:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763934.683230:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763934.683237:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763934.683252:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565923456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763934.683276:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763934.683281:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763934.683284:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1789763934.683293:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789763934.683301:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3869 00000100:00080000:2.0:1789763934.683307:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763934.683313:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763934.683321:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763934.683337:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565923584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763934.683400:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763934/real 1789763934] req@ffff93f8517c0a80 x1876699565923200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763934.683428:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565923200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763934.683443:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763934.683448:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763934.683454:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565923200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763934.683487:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565923328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763934.683516:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763934.683554:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763934/real 1789763934] req@ffff93f8517c1500 x1876699565923328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763934.683570:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565923328/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763934.683581:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763934.683585:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763934.683589:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565923328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763934.683596:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565923456:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763934.683619:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763934.683631:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763934/real 1789763934] req@ffff93f8517c0380 x1876699565923456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763934.683698:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565923456/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763934.683710:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763934.683714:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763934.683718:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565923456:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763934.683726:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565923584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763934.683763:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763934.683776:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763934/real 1789763934] req@ffff93f8517c2680 x1876699565923584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763934.683791:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565923584/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763934.683801:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763934.683805:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763934.683809:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565923584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763934.746848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763935.770918:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763936.794896:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763937.818888:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763938.842823:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789763939.802786:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763939.802799:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789763939.802805:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763939.802817:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3874 00000100:00080000:2.0:1789763939.802830:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763939.802840:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789763939.802890:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565923712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763939.802929:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763939.802937:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763939.802940:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763939.802945:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3874 00000100:00080000:2.0:1789763939.802949:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763939.802954:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763939.802965:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763939.802980:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565923840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763939.803018:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763939.803022:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789763939.803025:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763939.803030:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3874 00000100:00080000:2.0:1789763939.803035:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763939.803041:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789763939.803047:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789763939.803063:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565923968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763939.803076:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789763939.803081:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789763939.803084:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789763939.803089:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3874 00000100:00080000:2.0:1789763939.803092:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789763939.803100:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763939.803108:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565923712:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1789763939.803109:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763939.803128:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565924096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763939.803212:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763939.803279:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763939/real 1789763939] req@ffff93f8517c3b80 x1876699565923712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763939.803308:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565923712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763939.803324:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763939.803330:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763939.803335:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565923712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763939.803345:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565923840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763939.803373:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763939.803388:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763939/real 1789763939] req@ffff93f8517c0000 x1876699565923840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763939.803404:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565923840/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763939.803415:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763939.803419:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763939.803423:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565923840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763939.803430:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565923968:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763939.803449:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763939.803460:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763939/real 1789763939] req@ffff93f8517c1f80 x1876699565923968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763939.803474:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565923968/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763939.803493:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763939.803497:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763939.803501:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565923968:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763939.803536:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565924096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763939.803557:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763939.803568:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763939/real 1789763939] req@ffff93f8517c2300 x1876699565924096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763939.803582:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565924096/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763939.803592:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763939.803595:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763939.803599:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565924096:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763939.866928:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763940.891014:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763941.914848:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763942.938901:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763943.962899:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763944.922835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763944.922850:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763944.922856:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763944.922871:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3879 00000100:00080000:0.0:1789763944.922887:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763944.922899:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763944.922985:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40700 x1876699565924224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763944.923031:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763944.923038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763944.923042:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763944.923050:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3879 00000100:00080000:0.0:1789763944.923056:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763944.923063:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763944.923078:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763944.923105:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43b80 x1876699565924352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763944.923128:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763944.923140:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:3.0:1789763944.923141:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565924224:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763944.923146:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763944.923174:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3879 00000100:00080000:0.0:1789763944.923184:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763944.923194:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763944.923205:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763944.923234:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565924480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763944.923256:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763944.923263:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763944.923267:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1789763944.923280:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1789763944.923283:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3879 00000100:00080000:0.0:1789763944.923298:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763944.923310:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763944.923335:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763944.923355:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763944/real 1789763944] req@ffff93f850d40700 x1876699565924224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763944.923370:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565924608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789763944.923396:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40700 x1876699565924224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763944.923444:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763944.923452:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763944.923463:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565924224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763944.923502:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565924352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763944.923548:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763944.923575:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763944/real 1789763944] req@ffff93f850d43b80 x1876699565924352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763944.923602:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43b80 x1876699565924352/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763944.923621:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763944.923626:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763944.923631:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565924352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763944.923640:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565924480:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763944.923719:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763944.923742:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763944/real 1789763944] req@ffff93f850d40380 x1876699565924480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763944.923767:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565924480/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763944.923785:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763944.923792:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763944.923799:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565924480:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763944.923813:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565924608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763944.923850:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763944.923865:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763944/real 1789763944] req@ffff93f850d42300 x1876699565924608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763944.923880:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565924608/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763944.923891:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763944.923895:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763944.923899:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565924608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763944.986915:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763946.010888:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763947.034843:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763948.058915:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763949.082847:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763950.042743:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763950.042757:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763950.042762:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763950.042775:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3885 00000100:00080000:0.0:1789763950.042786:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763950.042796:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763950.042843:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565924736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763950.042879:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763950.042883:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763950.042887:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763950.042892:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3885 00000100:00080000:0.0:1789763950.042897:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763950.042901:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763950.042913:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763950.042967:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565924864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763950.042976:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763950.042979:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763950.042981:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763950.042985:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3885 00000100:00080000:0.0:1789763950.042989:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763950.042992:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763950.042997:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763950.043008:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565924736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763950.043009:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565924992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763950.043026:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763950.043029:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763950.043030:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763950.043033:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3885 00000100:00080000:0.0:1789763950.043035:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763950.043038:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763950.043041:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763950.043053:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565925120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1789763950.043081:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763950.043124:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763950/real 1789763950] req@ffff93f850d40000 x1876699565924736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763950.043141:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565924736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763950.043150:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763950.043155:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763950.043160:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565924736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763950.043166:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565924864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763950.043183:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763950.043193:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763950/real 1789763950] req@ffff93f850d40e00 x1876699565924864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763950.043203:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565924864/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763950.043209:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763950.043211:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763950.043214:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565924864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789763950.043234:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565924992:192.168.203.147@tcp:8:kworker.0 00000800:00000100:3.0:1789763950.043247:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763950.043254:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763950/real 1789763950] req@ffff93f850d42680 x1876699565924992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763950.043263:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565924992/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763950.043270:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763950.043272:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763950.043274:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565924992:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1789763950.043278:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565925120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1789763950.043290:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763950.043306:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763950/real 1789763950] req@ffff93f850d41c00 x1876699565925120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1789763950.043315:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565925120/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1789763950.043340:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1789763950.043344:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1789763950.043347:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565925120:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763950.106874:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763951.130785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763952.154838:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763953.178997:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763954.202844:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763955.162799:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763955.162810:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763955.162814:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763955.162823:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3890 00000100:00080000:3.0:1789763955.162834:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763955.162841:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763955.162899:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565925248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763955.162937:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763955.162941:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763955.162943:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763955.162948:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3890 00000100:00080000:3.0:1789763955.162953:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763955.162959:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763955.162969:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763955.162974:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565925248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763955.162986:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565925376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763955.162998:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763955.163001:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763955.163003:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763955.163007:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3890 00000100:00080000:3.0:1789763955.163010:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763955.163013:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763955.163018:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763955.163029:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565925504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763955.163038:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1789763955.163046:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789763955.163047:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763955.163052:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763955.163057:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3890 00000100:00080000:3.0:1789763955.163061:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763955.163066:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763955.163070:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763955.163081:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565925632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763955.163121:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763955/real 1789763955] req@ffff93f844d67800 x1876699565925248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763955.163149:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565925248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763955.163166:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763955.163171:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763955.163175:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565925248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763955.163202:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565925376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763955.163227:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763955.163240:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763955/real 1789763955] req@ffff93f844d67100 x1876699565925376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763955.163251:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565925376/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763955.163259:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763955.163262:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763955.163265:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565925376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763955.163270:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565925504:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763955.163286:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763955.163294:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763955/real 1789763955] req@ffff93f844d65c00 x1876699565925504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763955.163305:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565925504/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763955.163314:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763955.163318:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763955.163322:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565925504:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763955.163330:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565925632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763955.163358:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763955.163387:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763955/real 1789763955] req@ffff93f844d64e00 x1876699565925632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763955.163403:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565925632/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763955.163411:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763955.163414:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763955.163416:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565925632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763955.226940:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763956.250834:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763957.274832:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763958.298840:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763959.322950:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763960.283472:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763960.283502:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763960.283509:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763960.283525:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3895 00000100:00080000:3.0:1789763960.283540:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763960.283552:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763960.283605:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565925760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763960.283671:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763960.283676:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763960.283678:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763960.283684:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3895 00000100:00080000:3.0:1789763960.283691:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763960.283697:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763960.283727:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763960.283747:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565925888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763960.283761:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763960.283765:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763960.283767:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763960.283772:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3895 00000100:00080000:3.0:1789763960.283777:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763960.283781:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763960.283786:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763960.283817:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565926016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763960.283833:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763960.283839:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763960.283860:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763960.283867:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3895 00000100:00080000:3.0:1789763960.283873:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763960.283880:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763960.283888:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763960.283908:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565926144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789763960.283941:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565925760:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1789763960.284026:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763960.284095:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763960/real 1789763960] req@ffff93f844d65500 x1876699565925760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763960.284117:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565925760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763960.284129:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763960.284134:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763960.284138:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565925760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1789763960.284148:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565925888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763960.284175:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763960.284189:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763960/real 1789763960] req@ffff93f844d66a00 x1876699565925888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763960.284203:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565925888/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763960.284214:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763960.284218:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763960.284222:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565925888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1789763960.284228:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565926016:192.168.203.147@tcp:8:kworker.0 00000800:00000100:1.0:1789763960.284246:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763960.284256:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763960/real 1789763960] req@ffff93f844d67480 x1876699565926016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763960.284269:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565926016/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763960.284278:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763960.284281:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763960.284284:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565926016:192.168.203.147@tcp:8:kworker.0 00000100:00100000:1.0:1789763960.284325:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565926144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1789763960.284363:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1789763960.284385:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763960/real 1789763960] req@ffff93f844d66d80 x1876699565926144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1789763960.284401:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565926144/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1789763960.284411:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1789763960.284430:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1789763960.284435:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565926144:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763960.346921:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763961.370804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763962.394950:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763963.418885:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763964.442846:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763965.402862:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763965.402875:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763965.402882:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763965.402898:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3900 00000100:00080000:0.0:1789763965.402912:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763965.402923:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763965.402976:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565926272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763965.403018:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763965.403024:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763965.403027:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763965.403033:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3900 00000100:00080000:0.0:1789763965.403038:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763965.403045:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763965.403059:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763965.403092:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565926272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763965.403094:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565926400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763965.403128:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763965.403137:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763965.403143:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763965.403154:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3900 00000100:00080000:0.0:1789763965.403167:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763965.403179:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763965.403190:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789763965.403194:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763965.403228:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565926528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763965.403247:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763965.403253:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763965.403256:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763965.403264:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3900 00000100:00100000:2.0:1789763965.403283:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763965/real 1789763965] req@ffff93f850d41180 x1876699565926272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763965.403288:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763965.403305:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763965.403318:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763965.403343:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565926272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789763965.403345:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565926656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763965.403406:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763965.403417:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763965.403425:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565926272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763965.403459:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565926400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763965.403505:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763965.403523:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763965/real 1789763965] req@ffff93f850d43100 x1876699565926400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763965.403539:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565926400/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763965.403550:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763965.403555:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763965.403560:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565926400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763965.403569:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565926528:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763965.403595:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763965.403614:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763965/real 1789763965] req@ffff93f850d43800 x1876699565926528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763965.403632:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565926528/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763965.403690:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763965.403695:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763965.403702:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565926528:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763965.403711:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565926656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763965.403738:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763965.403756:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763965/real 1789763965] req@ffff93f850d41f80 x1876699565926656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763965.403772:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565926656/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763965.403783:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763965.403788:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763965.403793:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565926656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763965.466857:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763966.490910:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763967.514837:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763968.538919:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763969.562766:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763970.522827:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763970.522839:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763970.522845:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763970.522885:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3905 00000100:00080000:0.0:1789763970.522902:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763970.522913:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763970.522974:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43480 x1876699565926784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763970.523015:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763970.523020:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763970.523024:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763970.523032:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3905 00000100:00080000:0.0:1789763970.523037:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763970.523043:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763970.523058:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763970.523072:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565926784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763970.523087:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41500 x1876699565926912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763970.523104:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763970.523109:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763970.523112:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763970.523118:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3905 00000100:00080000:0.0:1789763970.523128:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763970.523133:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763970.523141:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:2.0:1789763970.523150:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789763970.523157:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41f80 x1876699565927040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763970.523175:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763970.523180:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763970.523183:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763970.523188:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3905 00000100:00080000:0.0:1789763970.523196:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1789763970.523202:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763970/real 1789763970] req@ffff93f850d43480 x1876699565926784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789763970.523206:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763970.523226:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763970.523228:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43480 x1876699565926784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789763995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763970.523249:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1789763970.523252:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43800 x1876699565927168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789763970.523258:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763970.523278:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565926784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763970.523331:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565926912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763970.523380:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763970.523395:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763970/real 1789763970] req@ffff93f850d41500 x1876699565926912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763970.523415:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41500 x1876699565926912/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763970.523422:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763970.523424:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763970.523427:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565926912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763970.523432:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565927040:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763970.523449:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763970.523456:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763970/real 1789763970] req@ffff93f850d41f80 x1876699565927040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763970.523464:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41f80 x1876699565927040/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789763995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763970.523470:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763970.523472:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763970.523474:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565927040:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763970.523478:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565927168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763970.523489:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763970.523495:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763970/real 1789763970] req@ffff93f850d43800 x1876699565927168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763970.523503:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43800 x1876699565927168/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789763995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763970.523508:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763970.523510:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763970.523512:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565927168:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763970.586864:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763971.610849:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763972.634845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763973.658787:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763974.682774:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763975.642834:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763975.642843:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763975.642847:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763975.642856:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3910 00000100:00080000:3.0:1789763975.642864:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763975.642869:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763975.642901:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565927296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763975.642923:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763975.642926:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763975.642928:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763975.642930:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3910 00000100:00080000:3.0:1789763975.642932:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763975.642935:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763975.642943:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763975.642952:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565927424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763975.642960:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565927296:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763975.642962:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763975.642970:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763975.642972:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763975.642975:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3910 00000100:00080000:3.0:1789763975.642977:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763975.642980:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763975.642983:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763975.642993:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565927552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763975.643000:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763975.643003:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763975.643004:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763975.643006:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3910 00000100:00080000:3.0:1789763975.643012:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763975.643014:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763975.643017:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763975.643024:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565927680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763975.643031:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763975.643074:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763975/real 1789763975] req@ffff93f844d64380 x1876699565927296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763975.643089:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565927296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763975.643100:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763975.643105:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763975.643110:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565927296:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763975.643134:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565927424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763975.643154:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763975.643163:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763975/real 1789763975] req@ffff93f844d64000 x1876699565927424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763975.643172:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565927424/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763975.643182:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763975.643184:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763975.643187:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565927424:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763975.643190:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565927552:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763975.643202:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763975.643208:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763975/real 1789763975] req@ffff93f844d64a80 x1876699565927552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763975.643215:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565927552/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763975.643221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763975.643222:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763975.643224:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565927552:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763975.643228:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565927680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763975.643238:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763975.643244:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763975/real 1789763975] req@ffff93f844d65f80 x1876699565927680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763975.643251:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565927680/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763975.643256:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763975.643257:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763975.643260:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565927680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763975.706803:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763976.730849:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763977.754820:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763978.778784:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763979.802855:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763980.762843:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763980.762855:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763980.762860:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763980.762871:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3915 00000100:00080000:0.0:1789763980.762881:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763980.762889:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763980.762930:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d43100 x1876699565927808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763980.762956:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763980.762959:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763980.762960:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763980.762963:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3915 00000100:00080000:0.0:1789763980.762966:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763980.762969:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763980.762978:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763980.762989:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565927808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789763980.762991:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41180 x1876699565927936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763980.763005:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763980.763007:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763980.763008:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763980.763010:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3915 00000100:00080000:0.0:1789763980.763013:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763980.763016:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763980.763020:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763980.763027:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d41c00 x1876699565928064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763980.763035:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763980.763038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763980.763040:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763980.763042:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3915 00000100:00080000:0.0:1789763980.763045:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763980.763047:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763980.763050:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763980.763057:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42680 x1876699565928192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763980.763115:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565927936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763980.763143:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565928064:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763980.763157:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565928192:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763980.826899:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763981.850774:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763982.874801:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1789763983.834895:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1789763983.834930:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1789763983.834943:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763983.834981:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763983.834989:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1789763983.834995:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1789763983.835020:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763980/real 1789763983] req@ffff93f850d41180 x1876699565927936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763983.835041:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41180 x1876699565927936/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763983.835054:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763983.835060:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763983.835064:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565927936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763983.835073:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763980/real 1789763983] req@ffff93f850d41c00 x1876699565928064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763983.835085:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d41c00 x1876699565928064/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763983.835094:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763983.835097:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763983.835100:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565928064:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763983.835107:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763980/real 1789763983] req@ffff93f850d42680 x1876699565928192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763983.835119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42680 x1876699565928192/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763983.835127:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763983.835130:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763983.835134:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565928192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763983.835141:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763980/real 1789763983] req@ffff93f850d43100 x1876699565927808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763983.835155:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d43100 x1876699565927808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763983.835163:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763983.835166:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763983.835168:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565927808:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789763983.898810:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763984.922794:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789763985.882737:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763985.882755:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1789763985.882760:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763985.882771:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3920 00000100:00080000:0.0:1789763985.882781:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763985.882789:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789763985.882831:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40e00 x1876699565928320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763985.882858:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763985.882861:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763985.882863:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763985.882866:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3920 00000100:00080000:0.0:1789763985.882869:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763985.882871:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763985.882887:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763985.882897:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40000 x1876699565928448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763985.882906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763985.882908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1789763985.882910:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789763985.882914:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565928320:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1789763985.882915:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3920 00000100:00080000:0.0:1789763985.882921:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763985.882927:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763985.882930:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:0.0:1789763985.882938:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d42300 x1876699565928576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1789763985.882945:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1789763985.882948:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1789763985.882949:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1789763985.882952:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3920 00000100:00080000:0.0:1789763985.882954:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1789763985.882956:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1789763985.882959:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789763985.882967:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f850d40380 x1876699565928704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763985.883008:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763985.883072:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763985/real 1789763985] req@ffff93f850d40e00 x1876699565928320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763985.883095:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40e00 x1876699565928320/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763985.883114:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763985.883119:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763985.883123:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565928320:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763985.883155:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565928448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763985.883181:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763985.883196:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763985/real 1789763985] req@ffff93f850d40000 x1876699565928448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763985.883208:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40000 x1876699565928448/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763985.883217:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763985.883221:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763985.883224:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565928448:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763985.883231:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565928576:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763985.883253:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763985.883265:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763985/real 1789763985] req@ffff93f850d42300 x1876699565928576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763985.883277:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d42300 x1876699565928576/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763985.883286:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763985.883290:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763985.883293:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565928576:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763985.883300:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565928704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763985.883335:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763985.883346:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763985/real 1789763985] req@ffff93f850d40380 x1876699565928704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763985.883359:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f850d40380 x1876699565928704/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763985.883369:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763985.883373:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763985.883377:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f850d40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565928704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763985.946804:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763986.970779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763987.994758:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763989.018773:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763990.042836:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763991.002755:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763991.002766:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763991.002772:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763991.002783:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3925 00000100:00080000:3.0:1789763991.002793:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763991.002801:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763991.002847:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565928832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763991.002875:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763991.002878:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763991.002880:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763991.002883:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3925 00000100:00080000:3.0:1789763991.002887:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763991.002890:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763991.002900:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789763991.002909:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565928832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789763991.002911:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565928960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763991.002923:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763991.002925:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763991.002927:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763991.002930:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3925 00000100:00080000:3.0:1789763991.002933:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763991.002935:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763991.002940:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763991.002950:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565929088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763991.002958:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763991.002961:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763991.002962:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763991.002964:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3925 00000100:00080000:3.0:1789763991.002974:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763991.002977:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763991.002980:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789763991.002988:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789763991.002989:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565929216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763991.003044:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763991/real 1789763991] req@ffff93f844d66300 x1876699565928832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763991.003061:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565928832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763991.003070:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763991.003073:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763991.003077:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565928832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763991.003102:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565928960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763991.003118:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763991.003127:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763991/real 1789763991] req@ffff93f844d65180 x1876699565928960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763991.003135:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565928960/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763991.003146:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763991.003149:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763991.003151:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565928960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763991.003155:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565929088:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763991.003166:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763991.003172:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763991/real 1789763991] req@ffff93f844d65f80 x1876699565929088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763991.003186:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565929088/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763991.003192:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763991.003194:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763991.003196:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565929088:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763991.003199:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565929216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763991.003211:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763991.003217:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763991/real 1789763991] req@ffff93f844d64a80 x1876699565929216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763991.003224:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565929216/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763991.003229:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763991.003231:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763991.003233:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565929216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763991.066777:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763992.090851:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763993.114783:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763994.138767:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763995.162915:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789763996.122741:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763996.122749:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789763996.122753:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763996.122762:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3931 00000100:00080000:3.0:1789763996.122771:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763996.122777:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789763996.122810:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565929344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763996.122838:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763996.122841:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763996.122842:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763996.122845:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3931 00000100:00080000:3.0:1789763996.122847:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763996.122850:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763996.122857:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763996.122865:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565929472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789763996.122873:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763996.122875:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789763996.122878:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763996.122881:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3931 00000100:00080000:3.0:1789763996.122883:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763996.122885:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763996.122888:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789763996.122899:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565929600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789763996.122903:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565929344:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789763996.122909:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789763996.122922:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789763996.122923:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789763996.122926:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3931 00000100:00080000:3.0:1789763996.122929:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789763996.122931:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789763996.122934:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789763996.122942:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565929728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789763996.123007:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763996.123072:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763996/real 1789763996] req@ffff93f844d64000 x1876699565929344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763996.123096:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565929344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763996.123110:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763996.123114:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763996.123119:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565929344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789763996.123128:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565929472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763996.123153:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763996.123167:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763996/real 1789763996] req@ffff93f844d64380 x1876699565929472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763996.123181:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565929472/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763996.123191:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763996.123195:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763996.123199:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565929472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789763996.123230:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565929600:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789763996.123255:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763996.123269:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763996/real 1789763996] req@ffff93f844d66d80 x1876699565929600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763996.123281:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565929600/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763996.123291:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763996.123295:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763996.123298:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565929600:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789763996.123304:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565929728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789763996.123326:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789763996.123339:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789763996/real 1789763996] req@ffff93f844d67480 x1876699565929728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789763996.123352:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565929728/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789763996.123361:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789763996.123364:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789763996.123367:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565929728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789763996.186877:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763997.210779:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763998.234803:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789763999.258780:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764000.282809:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789764001.242743:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764001.242754:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789764001.242758:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764001.242769:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3936 00000100:00080000:3.0:1789764001.242778:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764001.242787:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789764001.242822:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565929856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764001.242851:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764001.242854:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764001.242855:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764001.242858:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3936 00000100:00080000:3.0:1789764001.242861:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764001.242865:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764001.242872:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764001.242883:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565929984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789764001.242892:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565929856:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789764001.242892:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764001.242898:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789764001.242903:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764001.242907:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3936 00000100:00080000:3.0:1789764001.242909:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764001.242911:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764001.242914:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789764001.242924:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565930112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764001.242931:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764001.242933:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764001.242934:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764001.242936:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3936 00000100:00080000:3.0:1789764001.242938:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764001.242940:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764001.242943:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764001.242949:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565930240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789764001.242974:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764001.243020:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764001/real 1789764001] req@ffff93f844d66a00 x1876699565929856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764001.243034:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565929856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764001.243043:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764001.243047:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764001.243051:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565929856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789764001.243074:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565929984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764001.243096:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764001.243106:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764001/real 1789764001] req@ffff93f844d65500 x1876699565929984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764001.243115:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565929984/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764001.243121:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764001.243124:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764001.243128:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565929984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789764001.243133:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565930112:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789764001.243157:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764001.243171:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764001/real 1789764001] req@ffff93f844d64e00 x1876699565930112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764001.243186:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565930112/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764001.243196:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764001.243199:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764001.243202:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565930112:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789764001.243207:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565930240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764001.243226:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764001.243235:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764001/real 1789764001] req@ffff93f844d65c00 x1876699565930240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764001.243243:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565930240/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764001.243249:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764001.243251:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764001.243254:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565930240:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764001.306811:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764002.330805:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764003.354768:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764004.378793:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764005.402838:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789764006.362756:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764006.362770:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789764006.362775:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764006.362789:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3941 00000100:00080000:3.0:1789764006.362801:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764006.362810:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789764006.362855:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565930368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764006.362891:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764006.362895:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764006.362897:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764006.362901:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3941 00000100:00080000:3.0:1789764006.362906:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764006.362910:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764006.362923:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789764006.362930:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565930368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789764006.362937:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565930496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764006.362947:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764006.362950:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789764006.362952:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764006.362955:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3941 00000100:00080000:3.0:1789764006.362959:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764006.362962:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764006.362967:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789764006.362975:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565930624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764006.362985:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764006.362988:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764006.362989:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764006.362993:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3941 00000100:00080000:3.0:1789764006.362995:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764006.362999:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764006.363003:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789764006.363009:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789764006.363011:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565930752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789764006.363066:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764006/real 1789764006] req@ffff93f844d67100 x1876699565930368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764006.363084:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565930368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764006.363093:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764006.363097:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764006.363100:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565930368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789764006.363124:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565930496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764006.363143:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764006.363153:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764006/real 1789764006] req@ffff93f844d67800 x1876699565930496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764006.363162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565930496/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764006.363168:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764006.363170:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764006.363172:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565930496:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789764006.363176:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565930624:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789764006.363187:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764006.363194:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764006/real 1789764006] req@ffff93f844d64700 x1876699565930624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764006.363201:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565930624/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764006.363206:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764006.363208:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764006.363210:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565930624:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789764006.363213:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565930752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764006.363223:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764006.363229:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764006/real 1789764006] req@ffff93f844d67b80 x1876699565930752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764006.363236:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565930752/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764006.363246:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764006.363248:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764006.363250:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565930752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764006.426824:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764007.450819:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764008.474814:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764009.498823:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764010.522776:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789764011.482789:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764011.482806:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789764011.482813:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764011.482828:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3946 00000100:00080000:3.0:1789764011.482841:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764011.482852:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789764011.482907:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66680 x1876699565930880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764011.482946:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764011.482950:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764011.482952:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764011.482957:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3946 00000100:00080000:3.0:1789764011.482961:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764011.482966:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764011.482977:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764011.482992:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65880 x1876699565931008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764011.483003:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764011.483007:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789764011.483011:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1789764011.483013:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565930880:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789764011.483019:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3946 00000100:00080000:3.0:1789764011.483024:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764011.483029:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764011.483034:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789764011.483050:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67b80 x1876699565931136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764011.483061:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764011.483065:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764011.483067:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764011.483071:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3946 00000100:00080000:3.0:1789764011.483091:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764011.483096:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764011.483102:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789764011.483114:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789764011.483116:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64700 x1876699565931264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789764011.483175:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764011/real 1789764011] req@ffff93f844d66680 x1876699565930880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764011.483206:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66680 x1876699565930880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764011.483221:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764011.483227:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764011.483232:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565930880:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789764011.483261:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565931008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764011.483281:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764011.483291:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764011/real 1789764011] req@ffff93f844d65880 x1876699565931008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764011.483300:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65880 x1876699565931008/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764011.483306:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764011.483309:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764011.483311:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565931008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789764011.483316:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565931136:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789764011.483329:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764011.483335:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764011/real 1789764011] req@ffff93f844d67b80 x1876699565931136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764011.483343:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67b80 x1876699565931136/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764011.483348:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764011.483350:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764011.483352:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565931136:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789764011.483355:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565931264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764011.483365:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764011.483371:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764011/real 1789764011] req@ffff93f844d64700 x1876699565931264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764011.483378:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64700 x1876699565931264/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764011.483383:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764011.483385:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764011.483387:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565931264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764011.546842:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764012.570785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764013.594757:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764014.618866:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764015.642833:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789764016.602747:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764016.602760:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789764016.602766:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764016.602781:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3951 00000100:00080000:3.0:1789764016.602793:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764016.602803:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789764016.602852:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67800 x1876699565931392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764016.602884:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764016.602889:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764016.602891:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764016.602896:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3951 00000100:00080000:3.0:1789764016.602900:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764016.602905:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764016.602916:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764016.602932:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67100 x1876699565931520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789764016.602944:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565931392:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789764016.602952:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764016.602957:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789764016.602959:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764016.602965:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3951 00000100:00080000:3.0:1789764016.602969:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764016.602973:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764016.602979:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789764016.602994:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565931648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764016.603006:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764016.603010:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764016.603011:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764016.603016:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3951 00000100:00080000:3.0:1789764016.603020:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764016.603024:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764016.603030:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1789764016.603036:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789764016.603045:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565931776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789764016.603111:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764016/real 1789764016] req@ffff93f844d67800 x1876699565931392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764016.603135:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67800 x1876699565931392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764016.603148:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764016.603154:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764016.603159:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565931392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789764016.603189:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565931520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764016.603210:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764016.603220:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764016/real 1789764016] req@ffff93f844d67100 x1876699565931520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764016.603229:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67100 x1876699565931520/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764016.603235:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764016.603238:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764016.603240:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565931520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789764016.603244:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565931648:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789764016.603256:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764016.603262:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764016/real 1789764016] req@ffff93f844d65c00 x1876699565931648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764016.603270:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565931648/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764016.603280:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764016.603282:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764016.603284:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565931648:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789764016.603288:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565931776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764016.603298:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764016.603303:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764016/real 1789764016] req@ffff93f844d64e00 x1876699565931776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764016.603311:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565931776/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764016.603316:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764016.603318:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764016.603320:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565931776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764016.666854:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764017.690828:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764018.714849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764019.738860:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764020.762779:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789764021.722703:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764021.722714:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789764021.722719:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764021.722730:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3956 00000100:00080000:3.0:1789764021.722740:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764021.722750:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789764021.722781:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565931904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764021.722805:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764021.722808:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764021.722810:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764021.722813:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3956 00000100:00080000:3.0:1789764021.722816:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764021.722820:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764021.722827:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764021.722836:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565932032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764021.722847:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1789764021.722850:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565931904:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789764021.722852:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789764021.722856:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764021.722863:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3956 00000100:00080000:3.0:1789764021.722868:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764021.722872:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764021.722888:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789764021.722902:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565932160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764021.722915:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764021.722918:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764021.722920:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764021.722927:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3956 00000800:00000100:2.0:1789764021.722928:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1789764021.722935:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764021.722940:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764021.722953:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764021.722969:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565932288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789764021.722984:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764021/real 1789764021] req@ffff93f844d65500 x1876699565931904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764021.723005:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565931904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764021.723017:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764021.723022:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764021.723026:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565931904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789764021.723051:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565932032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764021.723074:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764021.723087:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764021/real 1789764021] req@ffff93f844d66a00 x1876699565932032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764021.723100:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565932032/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764021.723109:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764021.723113:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764021.723117:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565932032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789764021.723123:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565932160:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789764021.723140:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764021.723150:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764021/real 1789764021] req@ffff93f844d67480 x1876699565932160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764021.723162:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565932160/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764021.723171:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764021.723174:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764021.723177:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565932160:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789764021.723183:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565932288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764021.723200:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764021.723209:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764021/real 1789764021] req@ffff93f844d66d80 x1876699565932288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764021.723222:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565932288/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764021.723230:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764021.723233:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764021.723237:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565932288:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764021.786760:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764022.810808:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764023.834829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764024.858786:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764025.882854:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789764026.843542:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764026.843550:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789764026.843553:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764026.843561:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3961 00000100:00080000:3.0:1789764026.843571:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764026.843577:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789764026.843612:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565932416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764026.843668:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764026.843671:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764026.843673:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764026.843678:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3961 00000100:00080000:3.0:1789764026.843683:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764026.843687:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764026.843699:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764026.843715:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565932544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764026.843735:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764026.843738:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789764026.843740:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764026.843744:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3961 00000100:00080000:3.0:1789764026.843748:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764026.843753:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764026.843759:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789764026.843773:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565932672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764026.843783:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764026.843786:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764026.843788:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764026.843791:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3961 00000100:00080000:3.0:1789764026.843794:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764026.843798:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764026.843803:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764026.843815:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565932800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789764026.844093:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565932416:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1789764026.844180:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764026.844227:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764026/real 1789764026] req@ffff93f844d64380 x1876699565932416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764026.844247:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565932416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764026.844260:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764026.844274:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764026.844279:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565932416:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789764026.844286:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565932544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764026.844307:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764026.844319:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764026/real 1789764026] req@ffff93f844d64000 x1876699565932544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764026.844331:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565932544/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764026.844345:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764026.844348:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764026.844351:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565932544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789764026.844369:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565932672:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789764026.844389:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764026.844399:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764026/real 1789764026] req@ffff93f844d64a80 x1876699565932672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764026.844413:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565932672/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764026.844422:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764026.844425:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764026.844431:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565932672:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789764026.844437:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565932800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764026.844453:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764026.844462:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764026/real 1789764026] req@ffff93f844d65f80 x1876699565932800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764026.844473:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565932800/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764026.844481:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764026.844484:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764026.844488:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565932800:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764026.906915:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764027.931115:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764028.954857:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764029.979670:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764031.002849:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789764031.962708:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764031.962716:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789764031.962721:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764031.962729:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3966 00000100:00080000:3.0:1789764031.962741:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764031.962748:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789764031.962780:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65180 x1876699565932928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764031.962800:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764031.962802:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764031.962803:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764031.962806:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3966 00000100:00080000:3.0:1789764031.962809:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764031.962811:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764031.962818:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764031.962827:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66300 x1876699565933056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789764031.962838:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565932928:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789764031.962839:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764031.962849:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789764031.962854:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764031.962858:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3966 00000100:00080000:3.0:1789764031.962860:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764031.962861:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764031.962864:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789764031.962875:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65f80 x1876699565933184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764031.962881:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764031.962884:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764031.962885:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764031.962887:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3966 00000100:00080000:3.0:1789764031.962889:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764031.962891:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764031.962895:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764031.962901:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64a80 x1876699565933312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1789764031.962931:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764031.962978:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764031/real 1789764031] req@ffff93f844d65180 x1876699565932928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764031.962996:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65180 x1876699565932928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764031.963005:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764031.963008:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764031.963011:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565932928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789764031.963039:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565933056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764031.963055:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764031.963064:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764031/real 1789764031] req@ffff93f844d66300 x1876699565933056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764031.963071:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66300 x1876699565933056/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764031.963077:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764031.963079:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764031.963081:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565933056:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1789764031.963084:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565933184:192.168.203.147@tcp:8:kworker.0 00000800:00000100:2.0:1789764031.963095:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764031.963102:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764031/real 1789764031] req@ffff93f844d65f80 x1876699565933184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764031.963108:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65f80 x1876699565933184/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764031.963113:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764031.963115:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764031.963116:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565933184:192.168.203.147@tcp:8:kworker.0 00000100:00100000:2.0:1789764031.963120:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565933312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1789764031.963130:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1789764031.963136:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764031/real 1789764031] req@ffff93f844d64a80 x1876699565933312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789764031.963143:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64a80 x1876699565933312/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1789764031.963147:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1789764031.963149:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1789764031.963151:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565933312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764032.026814:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764033.050873:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764034.074804:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764035.098831:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764036.122848:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789764037.082728:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764037.082739:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789764037.082745:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764037.082755:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3972 00000100:00080000:3.0:1789764037.082767:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764037.082780:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789764037.082815:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64000 x1876699565933440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764037.082853:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764037.082857:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764037.082859:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764037.082865:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3972 00000100:00080000:3.0:1789764037.082869:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764037.082874:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764037.082884:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789764037.082891:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565933440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1789764037.082901:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64380 x1876699565933568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764037.082913:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764037.082917:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789764037.082919:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764037.082924:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3972 00000100:00080000:3.0:1789764037.082933:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764037.082938:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764037.082946:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000800:00000100:0.0:1789764037.082965:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789764037.082969:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66d80 x1876699565933696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764037.082985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764037.082988:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764037.082991:0:37:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764037.082995:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3972 00000100:00080000:3.0:1789764037.082999:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764037.083004:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1789764037.083011:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764037/real 1789764037] req@ffff93f844d64000 x1876699565933440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764037.083028:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64000 x1876699565933440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764037.083038:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764037.083042:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764037.083046:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565933440:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1789764037.083065:0:37:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789764037.083070:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565933568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1789764037.083080:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d67480 x1876699565933824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1789764037.083092:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789764037.083107:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764037/real 1789764037] req@ffff93f844d64380 x1876699565933568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764037.083119:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64380 x1876699565933568/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764037.083127:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764037.083130:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764037.083133:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565933568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789764037.083139:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565933696:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789764037.083156:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789764037.083165:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764037/real 1789764037] req@ffff93f844d66d80 x1876699565933696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764037.083176:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66d80 x1876699565933696/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764037.083183:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764037.083186:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764037.083189:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565933696:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789764037.083202:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565933824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789764037.083218:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789764037.083228:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764037/real 1789764037] req@ffff93f844d67480 x1876699565933824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764037.083239:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d67480 x1876699565933824/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764037.083247:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764037.083249:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764037.083252:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565933824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764037.146804:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764038.170779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764039.194808:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764040.218785:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764041.242873:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789764042.202789:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764042.202803:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1789764042.202809:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764042.202822:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3977 00000100:00080000:3.0:1789764042.202833:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764042.202841:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1789764042.202887:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d66a00 x1876699565933952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764042.202922:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764042.202926:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764042.202928:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764042.202932:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3977 00000100:00080000:3.0:1789764042.202936:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764042.202941:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764042.202951:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1789764042.202965:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65500 x1876699565934080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764042.202976:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1789764042.202981:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565933952:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1789764042.202981:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1789764042.202984:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764042.202992:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3977 00000100:00080000:3.0:1789764042.202996:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764042.203000:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764042.203006:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:3.0:1789764042.203018:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d64e00 x1876699565934208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1789764042.203029:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1789764042.203032:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1789764042.203034:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1789764042.203038:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3977 00000100:00080000:3.0:1789764042.203042:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1789764042.203047:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1789764042.203052:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1789764042.203062:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1789764042.203065:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f844d65c00 x1876699565934336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789764042.203121:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764042/real 1789764042] req@ffff93f844d66a00 x1876699565933952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764042.203151:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d66a00 x1876699565933952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764042.203161:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764042.203165:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764042.203170:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565933952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789764042.203202:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565934080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789764042.203223:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789764042.203235:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764042/real 1789764042] req@ffff93f844d65500 x1876699565934080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764042.203264:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65500 x1876699565934080/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764042.203273:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764042.203276:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764042.203279:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565934080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789764042.203285:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565934208:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789764042.203304:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789764042.203315:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764042/real 1789764042] req@ffff93f844d64e00 x1876699565934208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764042.203325:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d64e00 x1876699565934208/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764042.203339:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764042.203342:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764042.203346:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565934208:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789764042.203351:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565934336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789764042.203369:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789764042.203379:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764042/real 1789764042] req@ffff93f844d65c00 x1876699565934336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764042.203389:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f844d65c00 x1876699565934336/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764042.203397:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764042.203400:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764042.203403:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f844d65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565934336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764042.266824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764043.290845:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764044.314779:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764045.338824:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764046.362850:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789764047.322760:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789764047.322773:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789764047.322779:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789764047.322789:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3982 00000100:00080000:2.0:1789764047.322799:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789764047.322807:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789764047.322848:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2300 x1876699565934464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789764047.322871:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789764047.322875:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789764047.322877:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789764047.322881:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3982 00000100:00080000:2.0:1789764047.322885:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789764047.322889:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789764047.322900:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1789764047.322903:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565934464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1789764047.322912:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1f80 x1876699565934592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789764047.322921:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789764047.322925:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789764047.322926:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789764047.322929:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3982 00000100:00080000:2.0:1789764047.322932:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789764047.322935:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789764047.322940:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789764047.322947:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0000 x1876699565934720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789764047.322956:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789764047.322959:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789764047.322960:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789764047.322965:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3982 00000100:00080000:2.0:1789764047.322967:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789764047.322970:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789764047.322975:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789764047.322984:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c3b80 x1876699565934848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789764047.323013:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565934592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789764047.323044:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565934720:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789764047.323060:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565934848:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764047.386805:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764048.410829:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764049.434822:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1789764050.395017:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1789764050.395072:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1789764050.395092:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789764050.395142:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789764050.395153:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1789764050.395162:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1789764050.395215:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764047/real 1789764050] req@ffff93f8517c1f80 x1876699565934592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764050.395244:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1f80 x1876699565934592/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764050.395257:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764050.395262:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764050.395267:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565934592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789764050.395275:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764047/real 1789764050] req@ffff93f8517c0000 x1876699565934720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764050.395287:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0000 x1876699565934720/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764050.395364:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764050.395369:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764050.395372:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565934720:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789764050.395379:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764047/real 1789764050] req@ffff93f8517c3b80 x1876699565934848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764050.395390:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c3b80 x1876699565934848/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764050.395397:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764050.395401:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764050.395404:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565934848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789764050.395410:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764047/real 1789764050] req@ffff93f8517c2300 x1876699565934464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764050.395420:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2300 x1876699565934464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764050.395427:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764050.395430:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764050.395433:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565934464:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1789764050.458831:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764051.482846:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789764052.442782:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) af634c28-3647-4f73-bb24-40d7878858fb->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789764052.442791:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1789764052.442796:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789764052.442805:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 3987 00000100:00080000:2.0:1789764052.442815:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789764052.442823:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff93f844cbf000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1789764052.442860:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c2680 x1876699565934976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789764052.442887:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789764052.442891:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789764052.442893:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789764052.442896:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3987 00000100:00080000:2.0:1789764052.442900:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789764052.442904:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f851b31000: import ffff93f850c45000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789764052.442912:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789764052.442931:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0380 x1876699565935104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789764052.442937:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565934976:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1789764052.442947:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 1b6a8abb-fb57-4aa0-bd98-6434ab8cc627->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789764052.442951:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1789764052.442953:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789764052.442957:0:38:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: connect to NID 192.168.203.147@tcp last attempt 3987 00000100:00080000:2.0:1789764052.442961:0:38:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1789764052.442964:0:38:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff93f851b31000: import ffff93f850c41800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789764052.442969:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683244 ocd_grant: 28672, lost: 0. 00000100:00100000:2.0:1789764052.442979:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c1500 x1876699565935232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789764052.442989:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6a301392-4ca2-4c43-8653-61d4c47997a9->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1789764052.442993:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1789764052.442994:0:38:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1789764052.442997:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: connect to NID 192.168.203.147@tcp last attempt 3987 00000100:00080000:2.0:1789764052.443001:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1789764052.443005:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1789764052.443009:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff93f850b8d000: import ffff93f850b8a800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1789764052.443014:0:38:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683244 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1789764052.443024:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff93f8517c0a80 x1876699565935360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789764052.443059:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764052/real 1789764052] req@ffff93f8517c2680 x1876699565934976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764052.443082:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c2680 x1876699565934976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1789764077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764052.443093:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f844cbf000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764052.443097:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764052.443101:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:af634c28-3647-4f73-bb24-40d7878858fb:2356:1876699565934976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1789764052.443142:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565935104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789764052.443178:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789764052.443192:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764052/real 1789764052] req@ffff93f8517c0380 x1876699565935104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764052.443205:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0380 x1876699565935104/t0(0) o38->lustre-MDT0000-mdc-ffff93f851b31000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764052.443212:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c45000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764052.443215:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764052.443219:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565935104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1789764052.443226:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565935232:192.168.203.147@tcp:8:kworker.0 00000800:00000100:0.0:1789764052.443245:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789764052.443257:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764052/real 1789764052] req@ffff93f8517c1500 x1876699565935232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764052.443268:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c1500 x1876699565935232/t0(0) o8->lustre-OST0000-osc-ffff93f851b31000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 1 dl 1789764077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764052.443275:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850c41800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764052.443278:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764052.443281:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:1b6a8abb-fb57-4aa0-bd98-6434ab8cc627:2356:1876699565935232:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1789764052.443286:0:2356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565935360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1789764052.443305:0:2356:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1789764052.443315:0:2356:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1789764052/real 1789764052] req@ffff93f8517c0a80 x1876699565935360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789764052.443325:0:2356:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff93f8517c0a80 x1876699565935360/t0(0) o38->lustre-MDT0000-mdc-ffff93f850b8d000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1789764077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1789764052.443333:0:2356:0:(import.c:65:import_set_state_nolock()) ffff93f850b8a800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1789764052.443336:0:2356:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1789764052.443339:0:2356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff93f8517c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:6a301392-4ca2-4c43-8653-61d4c47997a9:2356:1876699565935360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1789764052.506828:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764053.530817:0:5443:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764054.554891:0:5444:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789764055.578853:0:11656:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 98314 lines, 98314 kept, 0 dropped, 0 bad.